Skip to content

Feature/v0.0.8#80

Merged
ravi-databricks merged 80 commits intomainfrom
feature/v0.0.8
Aug 6, 2024
Merged

Feature/v0.0.8#80
ravi-databricks merged 80 commits intomainfrom
feature/v0.0.8

Conversation

@ravi-databricks
Copy link
Contributor

@ravi-databricks ravi-databricks commented Jul 29, 2024

[v.0.0.8]

  • Added dlt append_flow api support: PR
  • Added dlt append_flow api support for silver layer: PR
  • Added support for file metadata columns for autoloader: PR
  • Added support for Bring your own custom transformation: Issue
  • Added support to Unify PyPI releases with GitHub OIDC: PR
  • Added demo for append_flow and file_metadata options: PR
  • Added Demo for silver fanout architecture: PR
  • Added documentation in docs site for new features: PR
  • Added unit tests to showcase silver layer fanout examples: PR
  • Fixed issue for No such file or directory: '/demo' :PR
  • Fixed issue DLT-META CLI onboard command issue for Azure: databricks.sdk.errors.platform.ResourceAlreadyExists :PR
  • Fixed issue Changed dbfs.create to mkdirs for CLI: PR
  • Fixed issue DLT-META CLI should use pypi lib instead of whl : PR

ravi-databricks and others added 30 commits April 30, 2024 12:28
1.dlt.append_flow api incorporation in bronze/silver layer
2.dlt.cdc_apply_changes additional api attributes incorporation
…-error

Corrected delta reader key from table to source_table
…mmand-issue-for-azure-databrickssdkerrorsplatformresourcealreadyexists

Changed dbfs.create to mkdirs for CLI
1.Custom Transformations for bronze layer
2.Unit tests
3.Increased version to v0.0.8
Fixing FileNotFoundError on onboarding file dir
Merging enhancement for issue: #49
Modified __version__ to match release version 0.0.8
…-fanout-architecture-in-silver-layer-using-dlt-meta

Added demo to showcase:

Silver fanout architecture
used cars input dataset containing rows for different countries
created 5 silver tables from single cars tables based on filter condition
Copy link

@neil90-db neil90-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, GJ Ravi!

@ravi-databricks ravi-databricks requested review from ganeshchand and removed request for ganeshchand August 5, 2024 18:45
@ravi-databricks ravi-databricks added the enhancement New feature or request label Aug 5, 2024
Copy link

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

Copy link

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

Copy link

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Submitting Approve.

@ravi-databricks ravi-databricks removed the request for review from ganeshchand August 5, 2024 19:08
Copy link

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

Copy link

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with write access.

Copy link

@ganeshchand ganeshchand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ravi-databricks ravi-databricks requested review from alexott, nfx and pohlposition and removed request for nfx August 6, 2024 17:09
@pohlposition
Copy link
Contributor

LGTM

Copy link
Contributor

@pohlposition pohlposition left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@pohlposition pohlposition left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Approved
Ship It
Launch

@ravi-databricks ravi-databricks merged commit 1bc01ba into main Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants