|
| 1 | +# SQL Ultimate Course |
1 | 2 |
|
2 |
| -# Data Warehouse and Analytics Project |
| 3 | +Welcome to the **SQL Ultimate Course** repository! 🚀 |
| 4 | +This course will cover everything you need to master SQL! |
3 | 5 |
|
4 |
| -Welcome to the **Data Warehouse and Analytics Project** repository! 🚀 |
5 |
| -This project demonstrates a comprehensive data warehousing and analytics solution, from building a data warehouse to generating actionable insights. Designed as a portfolio project, it highlights industry best practices in data engineering and analytics. |
6 |
| - |
7 |
| ---- |
8 |
| -## 🏗️ Data Architecture |
9 |
| - |
10 |
| -The data architecture for this project follows Medallion Architecture **Bronze**, **Silver**, and **Gold** layers: |
11 |
| - |
12 |
| - |
13 |
| -1. **Bronze Layer**: Stores raw data as-is from the source systems. Data is ingested from CSV Files into SQL Server Database. |
14 |
| -2. **Silver Layer**: This layer includes data cleansing, standardization, and normalization processes to prepare data for analysis. |
15 |
| -3. **Gold Layer**: Houses business-ready data modeled into a star schema required for reporting and analytics. |
16 |
| - |
17 |
| ---- |
18 |
| -## 📖 Project Overview |
19 |
| - |
20 |
| -This project involves: |
21 |
| - |
22 |
| -1. **Data Architecture**: Designing a Modern Data Warehouse Using Medallion Architecture **Bronze**, **Silver**, and **Gold** layers. |
23 |
| -2. **ETL Pipelines**: Extracting, transforming, and loading data from source systems into the warehouse. |
24 |
| -3. **Data Modeling**: Developing fact and dimension tables optimized for analytical queries. |
25 |
| -4. **Analytics & Reporting**: Creating SQL-based reports and dashboards for actionable insights. |
26 |
| - |
27 |
| -🎯 This repository is an excellent resource for professionals and students looking to showcase expertise in: |
28 |
| -- SQL Development |
29 |
| -- Data Architect |
30 |
| -- Data Engineering |
31 |
| -- ETL Pipeline Developer |
32 |
| -- Data Modeling |
33 |
| -- Data Analytics |
34 | 6 |
|
35 | 7 | ---
|
36 | 8 |
|
37 | 9 | ## 🛠️ Important Links & Tools:
|
38 | 10 |
|
39 | 11 | Everything is for Free!
|
40 |
| -- **[Datasets](datasets/):** Access to the project dataset (csv files). |
| 12 | +- **[Datasets](datasets/):** Access to the course dataset (csv files). |
41 | 13 | - **[SQL Server Express](https://www.microsoft.com/en-us/sql-server/sql-server-downloads):** Lightweight server for hosting your SQL database.
|
42 | 14 | - **[SQL Server Management Studio (SSMS)](https://learn.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms?view=sql-server-ver16):** GUI for managing and interacting with databases.
|
43 |
| -- **[Git Repository](https://github.com/):** Set up a GitHub account and repository to manage, version, and collaborate on your code efficiently. |
44 |
| -- **[DrawIO](https://www.drawio.com/):** Design data architecture, models, flows, and diagrams. |
45 |
| -- **[Notion](https://www.notion.com/):** All-in-one tool for project management and organization. |
46 |
| -- **[Notion Project Steps](https://thankful-pangolin-2ca.notion.site/SQL-Data-Warehouse-Project-16ed041640ef80489667cfe2f380b269?pvs=4):** Access to All Project Phases and Tasks. |
47 | 15 |
|
48 | 16 | ---
|
49 | 17 |
|
50 |
| -## 🚀 Project Requirements |
51 |
| - |
52 |
| -### Building the Data Warehouse (Data Engineering) |
53 |
| - |
54 |
| -#### Objective |
55 |
| -Develop a modern data warehouse using SQL Server to consolidate sales data, enabling analytical reporting and informed decision-making. |
56 |
| - |
57 |
| -#### Specifications |
58 |
| -- **Data Sources**: Import data from two source systems (ERP and CRM) provided as CSV files. |
59 |
| -- **Data Quality**: Cleanse and resolve data quality issues prior to analysis. |
60 |
| -- **Integration**: Combine both sources into a single, user-friendly data model designed for analytical queries. |
61 |
| -- **Scope**: Focus on the latest dataset only; historization of data is not required. |
62 |
| -- **Documentation**: Provide clear documentation of the data model to support both business stakeholders and analytics teams. |
63 |
| - |
64 |
| ---- |
65 |
| - |
66 |
| -### BI: Analytics & Reporting (Data Analysis) |
67 |
| - |
68 |
| -#### Objective |
69 |
| -Develop SQL-based analytics to deliver detailed insights into: |
70 |
| -- **Customer Behavior** |
71 |
| -- **Product Performance** |
72 |
| -- **Sales Trends** |
73 |
| - |
74 |
| -These insights empower stakeholders with key business metrics, enabling strategic decision-making. |
75 |
| - |
76 |
| -For more details, refer to [docs/requirements.md](docs/requirements.md). |
77 |
| - |
78 | 18 | ## 📂 Repository Structure
|
79 | 19 | ```
|
80 |
| -data-warehouse-project/ |
81 |
| -│ |
82 |
| -├── datasets/ # Raw datasets used for the project (ERP and CRM data) |
83 |
| -│ |
84 |
| -├── docs/ # Project documentation and architecture details |
85 |
| -│ ├── etl.drawio # Draw.io file shows all different techniquies and methods of ETL |
86 |
| -│ ├── data_architecture.drawio # Draw.io file shows the project's architecture |
87 |
| -│ ├── data_catalog.md # Catalog of datasets, including field descriptions and metadata |
88 |
| -│ ├── data_flow.drawio # Draw.io file for the data flow diagram |
89 |
| -│ ├── data_models.drawio # Draw.io file for data models (star schema) |
90 |
| -│ ├── naming-conventions.md # Consistent naming guidelines for tables, columns, and files |
| 20 | +sql-ultimate-course/ |
91 | 21 | │
|
92 |
| -├── scripts/ # SQL scripts for ETL and transformations |
93 |
| -│ ├── bronze/ # Scripts for extracting and loading raw data |
94 |
| -│ ├── silver/ # Scripts for cleaning and transforming data |
95 |
| -│ ├── gold/ # Scripts for creating analytical models |
| 22 | +├── datasets/ # Raw datasets used for the course (ERP and CRM data) |
96 | 23 | │
|
97 |
| -├── tests/ # Test scripts and quality files |
| 24 | +├── docs/ # course documentation and architecture details |
98 | 25 | │
|
99 |
| -├── README.md # Project overview and instructions |
100 |
| -├── LICENSE # License information for the repository |
101 |
| -├── .gitignore # Files and directories to be ignored by Git |
102 |
| -└── requirements.txt # Dependencies and requirements for the project |
| 26 | +└── scripts/ # All codes presented in the course |
103 | 27 | ```
|
104 | 28 | ---
|
105 | 29 |
|
106 | 30 |
|
107 | 31 | ## 🛡️ License
|
108 | 32 |
|
109 |
| -This project is licensed under the [MIT License](LICENSE). You are free to use, modify, and share this project with proper attribution. |
| 33 | +This course is licensed under the [MIT License](LICENSE). You are free to use, modify, and share this project with proper attribution. |
110 | 34 |
|
111 | 35 | ## 🌟 About Me
|
112 | 36 |
|
|
0 commit comments