A production-ready PySpark project template with medallion architecture, Python packaging, unit tests, integration tests, CI/CD automation, Databricks Asset Bundles, and DQX data quality framework.

Production-grade Databricks infrastructure templates for Azure. Deploy in 20 minutes with VNet injection, Unity Catalog, managed identity. Perfect for learning and prototyping. Free and open source.

Metadata-driven framework for Databricks Spark Declarative Pipelines. Config-driven, pattern based approach to batch & streaming across the medallion architecture. Deploys via Declarative Automation Bundles. Built for simplicity, extensibility, and alignment with the Databricks product roadmap.

Example Python project that uses uv for dependency management and Databricks Asset Bundles for deployment.

databricks-dab-lab is an end-to-end lab that shows how to deploy Databricks Asset Bundles (DABs) with GitHub Actions, using Terraform to provision an Azure Databricks workspace + cluster, then deploying and running three jobs in sequence (data setup → ETL → ML training).

Production-ready support ticket classification using Unity Catalog AI Functions, Vector Search, and RAG. Features 6-phase workflow, knowledge base integration, and Streamlit dashboard.

Medallion Architecture for Data Engineering projects

Restaurant Chain Analytics Lakehouse with Data Vault 2.1 architecture and dimensional modeling using star schema via Lakeflow Spark Declarative Pipelines in Azure Databricks

Automate Databricks Apps deployments with GitHub Actions and Databricks Asset Bundles (DABs).

End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.

Different examples around Databricks Asset Bundsls (DABs)

A local databricks asset bundle inspector to help you inspect your DABs before deployment

Databricks Pipeline based on Ceres metadata Index

Databricks-native data trust pipeline — intake certification, drift gating, and control benchmarking in a single deployable product.

Real Estate ELT pipeline using Databricks Asset Bundles on GCP. Ingests, transforms, and analyzes property data via Delta Live Tables. Follows medallion architecture (Bronze/Silver/Gold), modular Python design, CI/CD automation with GitHub Actions, and full Unit and Integration tests coverage.

Databricks-native intelligent data transformation engine — coherence-scored Bronze/Silver/Gold with entity resolution and temporal reconciliation in a single deployable product.

Production-ready Databricks Asset Bundle for cross-region ML model serving using Delta Sharing. Deploy models and feature tables across workspaces with zero-copy data access and automated online feature store sync.

Portfólio de projetos práticos de Data Engineering e ML com Databricks — Delta Lake, MLflow, DLT, dbt, Kafka e mais

Databricks hands on projects