amadou/SecondBrain
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

vault backup: 2026-07-18 23:06:13

Browse Source
This commit is contained in:
amadou
2026-07-18 23:06:13 +02:00
parent 0013290871
commit fda2206f43
24 changed files with 3116 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
10 Knowledge/CTO Academy/Data/Data Lake vs Data Warehouse vs Lakehouse.md
+499
View File
@@ -0,0 +1,499 @@
**Time: ~10–15 minutes**
Today's goal is that if someone says:
> "The data is in the lake."
or:
> "We're moving to a lakehouse."
or:
> "That's handled by Synapse."
you immediately understand what category of thing they're talking about.
---
# 1. Start with the simplest mental model
Think of these as answering different questions:
|Concept|Think of it as|
|---|---|
|**Data Lake**|A place to store huge amounts of data|
|**Data Warehouse**|A structured system optimized for analytics|
|**Lakehouse**|An architecture combining characteristics of both|
|**Azure Synapse**|A Microsoft analytics platform providing tools to work with data|
The first three describe **data architectures/storage approaches**.
Synapse is a **product/platform**.
That's the first distinction to remember.
---
# 2. Data Lake 🌊
A Data Lake is designed to store **large quantities of data in many forms**.
You might put:
```text
GA4 event exports
JW Player logs
JSON files
CSV files
Parquet files
Application logs
Images
Video metadata
IoT data
Database exports
```
into a Data Lake.
The important thing is that the data doesn't necessarily need to be modeled for business reporting before you store it.
Think:
> **Store first. Figure out how to use it later.**
For your company, something like Azure Data Lake Storage could contain enormous quantities of raw and processed data.
You could potentially organize it:
```text
Data Lake
├── bronze/
│ ├── ga4/
│ ├── jwplayer/
│ └── ott/
│
├── silver/
│ ├── audience/
│ └── content/
│
└── gold/
├── audience_metrics/
└── content_performance/
```
So remember our previous lesson:
**A Data Lake can contain Bronze, Silver AND Gold.**
The Lake is **where/how data is stored**.
The Medallion layers describe **the maturity of the data**.
---
# 3. Data Warehouse 🏢
A Data Warehouse is much more structured.
It's designed primarily for:
> **Analytics and reporting.**
Instead of throwing everything into it, you intentionally organize data for analysis.
Imagine:
```text
DIM_DATE
│
│
DIM_CONTENT ─── FACT_VIEWING ─── DIM_USER
│
│
DIM_PLATFORM
```
Now you can easily ask:
> How many hours were watched...
> by content...
> by platform...
> by month...
> by audience?
That's what warehouses are excellent at.
You'll eventually hear terms such as:
- Fact tables
- Dimension tables
- Star schemas
- Snowflake schemas
Those are closely associated with analytical data modeling and warehouses.
We'll cover those soon.
---
# 4. Data Lake vs Data Warehouse
The traditional distinction looks roughly like this:
||Data Lake|Data Warehouse|
|---|---|---|
|Data|Raw + processed|Primarily structured|
|Structure|Flexible|Highly organized|
|Cost|Generally cheaper storage|Generally more expensive compute/storage|
|Purpose|Many possible uses|Analytics/reporting|
|Users|Data engineers/scientists|Analysts/BI|
|Schema|Often applied later|Typically defined deliberately|
A simplified architecture could therefore be:
```text
SOURCE SYSTEMS
│
▼
DATA LAKE
│
Bronze
│
Silver
│
▼
DATA WAREHOUSE
│
Gold
│
▼
Power BI
```
Again, that's **one possible architecture**, not a universal rule.
---
# 5. The problem
Companies started asking:
> Why are we maintaining two completely separate worlds?
They had:
```text
Data Lake
AND
Data Warehouse
```
The lake was flexible and inexpensive.
The warehouse was structured and excellent for analytics.
But moving data between them added:
- complexity
- duplication
- pipelines
- governance challenges
- additional cost
This contributed to the rise of the:
# Lakehouse 🏠🌊
The idea is:
> **Can we get warehouse-like capabilities directly on top of lake-style storage?**
Conceptually:
```text
LAKEHOUSE
┌────────────────────────────┐
│ │
│ Bronze │
│ ↓ │
│ Silver │
│ ↓ │
│ Gold │
│ │
│ Structured + Unstructured │
│ │
│ BI + ML + Data Science │
│ │
└────────────────────────────┘
│
▼
Power BI
```
Technologies such as Delta Lake helped make this architecture practical by adding database-like capabilities to data stored in a lake.
The big idea:
> **Data Lake flexibility + Data Warehouse capabilities = Lakehouse**
That's simplified, but it's the mental model I want you to remember.
---
# 6. Where does Azure Synapse fit?
This is where people get confused.
Azure Synapse Analytics is not simply a Data Lake.
It's not simply a Data Warehouse either.
It's a Microsoft **analytics platform**.
It provides capabilities around things such as:
```text
Data
↓
Pipelines
↓
Transformations
↓
SQL
↓
Spark
↓
Analytics
```
It can interact with services such as Azure Data Lake Storage and provide SQL and Spark-based analytics capabilities.
So someone saying:
> "Our data is in Synapse."
is a little like saying:
> "Our website is in AWS."
Okay...
But **where and how?**
The CTO question becomes:
> "Can you show me how we're using Synapse in our architecture?"
Maybe they're using:
- Synapse pipelines for data movement.
- Spark for transformations.
- Serverless SQL to query files in the Data Lake.
- Dedicated SQL pools as a warehouse.
Those are very different things.
---
# 7. Connecting Lessons 1 and 2
Now we have two separate dimensions.
### Data maturity
```text
Bronze
↓
Silver
↓
Gold
```
### Data architecture
```text
Data Lake
Data Warehouse
Lakehouse
```
And then we have a product:
```text
Azure Synapse
```
These concepts can overlap.
For example:
```text
Azure Platform
│
▼
Azure Data Lake Storage
Bronze → Silver → Gold
│
▼
Synapse SQL
│
▼
Power BI
```
That's a perfectly plausible architecture.
But your actual company architecture could be different.
---
# 8. CTO perspective
Imagine you're in a meeting and someone says:
> "We need a Lakehouse."
Don't immediately say yes.
Ask:
> **"What problem are we solving that our current Data Lake and Warehouse architecture can't solve?"**
That's the executive question.
Because "Lakehouse" sounds modern.
That doesn't automatically make it necessary.
Then ask:
> "Where does our Gold layer currently live?"
> "Are we duplicating data between our Lake and Warehouse?"
> "What's driving our current cost?"
> "What workloads are we supporting: BI, data science, ML, or all three?"
> "How is data governed across these environments?"
Now you're participating in an architecture discussion without needing to write a single Spark job.
---
# 9. Meeting scenario
A data architect tells you:
> "We should migrate our architecture to a Lakehouse."
A weaker response would be:
> "Okay, what's involved?"
A stronger CTO response:
> **"Before we discuss the technology, I'd like to understand the problem we're solving. Are we trying to reduce duplication between the Lake and Warehouse, improve performance, lower cost, or support new workloads? Once we agree on the problem, we can determine whether a Lakehouse is actually the right architecture."**
Notice our executive communication training appearing again?
**Control the structure.**
You just changed:
> Should we implement a Lakehouse?
into:
> What problem are we trying to solve?
That's exactly the skill you told me you want to develop.
---
# 10. Your cheat sheet
Remember these four sentences:
> 🌊 **Data Lake:** Store large amounts of diverse data flexibly.
> 🏢 **Data Warehouse:** Organize structured data for analytics and reporting.
> 🏠 **Lakehouse:** Bring warehouse-like capabilities to lake-style storage.
> 🧰 **Azure Synapse:** Microsoft's analytics platform that provides multiple tools for working with and analyzing data.
And remember:
> **Bronze/Silver/Gold tells me how refined the data is.**
> **Lake/Warehouse/Lakehouse tells me about the architecture.**
> **Synapse tells me about the technology/platform being used.**
That's the distinction I want you to walk away with tonight.
---
### Obsidian
Create:
```text
Knowledge
└── CTO Academy
└── Data
└── Data Lake vs Data Warehouse vs Lakehouse.md
```
At the bottom:
```markdown
## Related Concepts
- [[Medallion Architecture]]
- [[Azure Synapse]]
- [[ETL vs ELT]]
- [[Star Schema]]
- [[Fact Table]]
- [[Dimension Table]]
- [[Data Lake]]
- [[Data Warehouse]]
- [[Lakehouse]]
```
Don't create individual pages for all of them yet.
**Lesson 3 should be ETL vs ELT.** That's the missing bridge between today's lesson and the first lesson: _how does the data actually travel from the source through Bronze, Silver and Gold?_
10 Knowledge/CTO Academy/Data/ETL vs ELT.md
+757
View File
@@ -0,0 +1,757 @@
## How Data Moves Through the Architecture
**Time: ~10–15 minutes**
Today we're connecting [[Medallion Architecture]] and [[Data Lake vs Data Warehouse vs Lakehouse]]
.
You now understand:
```text
Lesson 1
Bronze → Silver → Gold
Data maturity
Lesson 2
Data Lake → Data Warehouse → Lakehouse
Data architecture
Lesson 3
ETL / ELT
How data gets there and gets transformed
```
By the end of this lesson, when someone says:
> "We're using an ELT pipeline."
you should understand exactly what they mean and why they might have chosen it.
---
# 1. The simplest definition
Both ETL and ELT describe processes for moving and transforming data.
The letters mean:
```text
E = Extract
T = Transform
L = Load
```
The difference is **the order**.
### ETL
```text
Extract
↓
Transform
↓
Load
```
### ELT
```text
Extract
↓
Load
↓
Transform
```
That tiny difference represents an important architectural decision.
---
# 2. ETL
## Extract → Transform → Load
Imagine we want data from JW Player.
We first:
### Extract
Get the data from JW Player.
```text
JW Player API
↓
Raw viewing data
```
Then:
### Transform
Before putting it into our final destination, we:
- remove duplicates
- fix dates
- standardize IDs
- remove invalid records
- calculate fields
- join datasets
Then:
### Load
We put the transformed data into the destination.
```text
JW Player
↓
EXTRACT
↓
Raw data
↓
TRANSFORM
↓
Clean data
↓
LOAD
↓
Data Warehouse
```
The important idea:
> **The transformation happens BEFORE the data reaches its destination.**
---
# 3. Why was ETL traditionally popular?
Historically, data warehouses were:
- expensive
- highly structured
- limited in storage
- optimized for specific analytical workloads
You didn't want to dump everything into them.
You wanted to clean the data first.
Think of it like moving into a small apartment.
You don't bring everything you've accumulated over 20 years.
You:
```text
Pack
↓
Sort
↓
Throw things away
↓
Move
```
That's ETL.
You decide what is useful **before loading it into the destination**.
---
# 4. ELT
Modern cloud platforms changed things.
Storage became much cheaper.
Compute became much more scalable.
So organizations started saying:
> Why don't we load the raw data first and transform it afterward?
That's:
## Extract → Load → Transform
Example:
```text
JW Player
↓
EXTRACT
↓
LOAD
↓
Data Lake / Warehouse
↓
TRANSFORM
↓
Cleaned data
```
The key difference:
> **We preserve the raw data in the destination before transforming it.**
---
# 5. Your media-company example
Imagine you receive **100 million JW Player events**.
With traditional ETL:
```text
JW Player
↓
Extract
↓
Transformation Server
↓
Clean / Aggregate
↓
Data Warehouse
```
You might only load the processed results.
Perhaps:
```text
Show A → 50,000 views
Show B → 25,000 views
```
But what happens six months later when someone asks:
> "Can we calculate average viewing duration differently?"
Potential problem.
Maybe the original detailed data wasn't preserved.
---
With ELT:
```text
JW Player
↓
Extract
↓
Data Lake
↓
RAW DATA STORED
↓
Transform
↓
Silver
↓
Transform
↓
Gold
```
Now you still have the original events.
You can go back.
Reprocess.
Create new metrics.
Investigate problems.
That's a major advantage.
---
# 6. Connecting this to Medallion Architecture
This is where everything we've learned starts connecting.
A modern architecture might look like:
```text
SOURCE SYSTEMS
GA4
JW Player
CRM
Finance
OTT
Apps
│
│ EXTRACT
▼
┌─────────────────┐
│ BRONZE │
│ │
│ RAW DATA │
└────────┬────────┘
│
│ TRANSFORM
▼
┌─────────────────┐
│ SILVER │
│ │
│ CLEAN DATA │
└────────┬────────┘
│
│ TRANSFORM
▼
┌─────────────────┐
│ GOLD │
│ │
│ BUSINESS READY │
└────────┬────────┘
│
▼
Power BI
```
This architecture is conceptually closer to **ELT**.
Why?
Because we:
```text
Extract
↓
Load raw data
↓
Transform afterward
```
The transformations create increasingly refined datasets.
---
# 7. ETL vs ELT
Here's your executive-level comparison:
||ETL|ELT|
|---|---|---|
|Order|Extract → Transform → Load|Extract → Load → Transform|
|Raw data stored first|Not necessarily|Usually|
|Transformation location|Before destination|In/around destination platform|
|Flexibility|Lower|Higher|
|Reprocessing|Potentially harder|Easier if raw data retained|
|Traditional warehouses|Common|Less traditional|
|Modern cloud data platforms|Used|Very common|
Neither is automatically better.
Architecture depends on:
- data volume
- cost
- security
- privacy
- latency
- technology
- business requirements
---
# 8. A subtle but important point
People often use the word:
> **Pipeline**
A data pipeline is simply an automated process that moves and/or transforms data.
For example:
```text
JW Player API
↓
Pipeline
↓
Bronze
```
Another pipeline might do:
```text
Bronze
↓
Pipeline
↓
Silver
```
Another:
```text
Silver
↓
Pipeline
↓
Gold
```
So your entire data architecture might contain dozens or hundreds of pipelines.
Some run:
```text
Every 5 minutes
```
Some:
```text
Hourly
```
Some:
```text
Nightly
```
Some:
```text
Real-time
```
---
# 9. Batch vs Streaming
This is related, but it's a separate concept we'll study later.
For now, just understand:
### Batch
Data moves periodically.
```text
Every night at 2 AM:
Yesterday's JW Player data
↓
Pipeline
↓
Data Lake
```
### Streaming
Data continuously flows.
```text
User watches video
↓
Event
↓
Pipeline
↓
Data Platform
```
Think:
> **Batch = packages of data periodically.**
> **Streaming = continuous flow of events.**
This distinction can become important for a media company.
---
# 10. Where does Azure Synapse fit?
Remember Lesson 2:
Synapse is a **platform**.
A company could potentially use Synapse capabilities to orchestrate pipelines that:
```text
Extract
↓
Load
↓
Transform
```
For example:
```text
JW Player
↓
Synapse Pipeline
↓
Azure Data Lake
↓
Bronze
↓
Transformation
↓
Silver
↓
Transformation
↓
Gold
↓
Power BI
```
Again, your company's actual architecture might differ.
When you're back from vacation, that's something worth mapping.
---
# 11. CTO perspective
Imagine an architect says:
> "We're moving from ETL to ELT."
Don't just respond:
> "Okay."
You want to understand **why**.
You could ask:
> "What's driving the change?"
Then:
> "Are we preserving all raw source data?"
> "Where are transformations executed?"
> "What's the impact on compute costs?"
> "How are transformations tested?"
> "How do we handle schema changes from source systems?"
> "Can we reprocess historical data if business rules change?"
These questions reveal the architectural consequences.
---
# 12. Meeting scenario
Imagine your analyst says:
> "We need to change the way we calculate total viewing hours, but we'd have to reload everything."
Instead of immediately discussing implementation, you could ask:
> **"Do we still have the original raw viewing events in our Bronze layer?"**
If yes:
> **"Then can we reprocess the affected data through Silver and regenerate the Gold dataset using the new business rule?"**
Now you're thinking architecturally.
You understand:
```text
Bronze
Original truth
↓
Silver
Clean data
↓
Gold
Business interpretation
```
If the **business rule** changes, you may not need new source data.
You may need to **reprocess existing data**.
That's one reason retaining raw data can be valuable.
---
# 13. Another CTO-level issue: Garbage In, Garbage Out
ELT has a potential downside.
Because storing raw data is easy, organizations sometimes end up with:
```text
Data Lake
↓
More Data
↓
More Data
↓
More Data
↓
Nobody knows what anything means
```
This is sometimes jokingly called a:
> **Data Swamp**
A Data Lake without proper:
- governance
- metadata
- ownership
- documentation
- lineage
- quality controls
can become difficult to use.
This connects directly to your role in **Data Governance**.
The technology doesn't create trustworthy data.
Governance does.
---
# 14. Your executive communication exercise
Someone says in a meeting:
> "We should move everything to ELT because it's the modern approach."
Your response could be:
> **"Before we decide on ETL versus ELT, I'd like to understand the problem we're solving. If the goal is to preserve raw data and give us more flexibility to reprocess it as business requirements change, ELT may make sense. But we should also understand the impact on storage, compute, governance and data quality before changing the architecture."**
Notice the pattern again:
**Clarify the problem → acknowledge the option → identify tradeoffs → guide the decision.**
You don't need to be the data engineer.
You need to make sure the organization is solving the right problem.
---
# Your Cheat Sheet
Remember this:
> **ETL:** Clean it, then load it.
> **ELT:** Load it, then clean it.
> **Pipeline:** Automated movement/transformation of data.
> **Batch:** Move data periodically.
> **Streaming:** Move data continuously.
And the connection between our first three lessons:
```text
SOURCE SYSTEMS
│
│
Extract / Load
│
▼
🥉 BRONZE
Raw Data
│
Transform
│
▼
🥈 SILVER
Clean Data
│
Transform
│
▼
🥇 GOLD
Business Data
│
▼
Power BI
```
That's the mental model I want you to keep.
## Add to Obsidian
Create:
```text
Knowledge
└── CTO Academy
└── Data
└── ETL vs ELT.md
```
At the bottom:
```markdown
## Related Concepts
- [[Medallion Architecture]]
- [[Data Lake]]
- [[Data Warehouse]]
- [[Lakehouse]]
- [[Azure Synapse]]
- [[Data Pipeline]]
- [[Batch Processing]]
- [[Stream Processing]]
- [[Data Governance]]
- [[Data Lineage]]
```
### Next: Lesson 4
I think we're ready to move **inside the Gold layer**.
**Fact Tables & Dimension Tables.**
You've encountered these concepts before when we were playing with Metabase and discussing grain/granularité, so Lesson 4 should connect something you already partially understand with the architecture we're building now:
**Source → ELT → Bronze → Silver → Gold → Fact/Dimensions → Power BI.**
That's when this whole data architecture should really start clicking together.
10 Knowledge/CTO Academy/Data/Medallion Architecture.md
+405
View File
@@ -0,0 +1,405 @@
## Bronze, Silver & Gold
**Time: ~10 minutes**
### 1. Definition
**Medallion Architecture** is a way of organizing data into layers based on how processed and trustworthy it is.
The three common layers are:
```text
DATA SOURCES
↓
┌──────────────┐
│ BRONZE │ Raw data
└──────────────┘
↓
┌──────────────┐
│ SILVER │ Cleaned & standardized
└──────────────┘
↓
┌──────────────┐
│ GOLD │ Business-ready
└──────────────┘
↓
Power BI / Analytics / KPIs / Applications
```
The fundamental idea is:
> **Raw → Clean → Business-ready**
---
# 2. Why does this exist?
Imagine your company collects data from:
```text
GA4
JW Player
Cable/linear TV
Advertising
CRM
Finance
HR
CMS
Mobile apps
OTT apps
```
Every system has different formats, naming conventions and levels of quality.
You don't want your Power BI analysts independently cleaning and interpreting all of that data every time they create a report.
Instead, you progressively transform the data.
---
# 🥉 Bronze: "What did we receive?"
Bronze is the **raw data**.
Ideally, you preserve the data approximately as it arrived from the source.
Example from JW Player:
```text
user_id video_id play_time timestamp
82372 VID123 134 2026-07-18 12:34
82373 VID555 NULL 2026-07-18 12:35
82372 VID123 134 2026-07-18 12:34
```
Notice:
- NULL values
- duplicates
- potentially incorrect data
That's okay.
Bronze isn't necessarily supposed to be beautiful.
### Why keep it?
Because if your transformation is wrong, you can go back to the original data.
Think:
> **Bronze = source of historical truth about what we received.**
---
# 🥈 Silver: "What data can we trust technically?"
Now we clean it.
We might:
- Remove duplicates
- Standardize dates
- Handle NULL values
- Validate IDs
- Standardize country codes
- Join related datasets
- Correct data types
Our data becomes:
```text
user_id video_id watch_seconds date
82372 VID123 134 2026-07-18
```
Now it's consistent.
But here's an important distinction:
**Silver doesn't necessarily understand the business.**
It understands the **data**.
Think:
> **Silver = clean and standardized data.**
---
# 🥇 Gold: "What does the business need?"
This is where your original question comes in.
Gold data is designed for **business consumption**.
Instead of millions of video events, you might have:
```text
content viewers watch_hours
Show A 152,000 47,500
Show B 89,000 31,200
Show C 67,000 19,800
```
Or:
```text
month streaming_hours unique_viewers
January 1,500,000 320,000
February 1,700,000 350,000
March 1,850,000 380,000
```
Now Power BI can consume this directly.
Think:
> **Gold = data organized around business questions.**
---
# 3. Your TV-company example
Let's imagine your CEO asks:
> "How many people watched our content last month across all digital platforms?"
The architecture could look like this:
```text
JW Player ──────────┐
│
GA4 ────────────────┤
│
Mobile App ─────────┤
↓
BRONZE
Raw source data
↓
SILVER
Clean IDs / Remove duplicates
Standardize timestamps
Match content identifiers
↓
GOLD
Monthly Content Audience
↓
Power BI
↓
CEO
```
Your CEO should never have to understand Bronze.
Your Power BI developer ideally shouldn't repeatedly rebuild Silver transformations.
The Gold layer should provide trusted business-ready datasets.
---
# 4. The CTO perspective
Here's where **your role** becomes important.
You don't necessarily need to know how your data engineer writes the transformation.
You need to ask questions like:
> **Who owns the Gold layer?**
> **Who defines the business rules?**
> **How do we know these KPIs are correct?**
> **Can we trace a Gold KPI back to its original source?**
> **What happens when the source changes?**
> **Are Power BI analysts calculating KPIs independently, or consuming governed metrics?**
That last question is particularly relevant to your Data Governance responsibilities.
Imagine:
```text
Power BI Analyst A
"Active User = logged-in user"
Power BI Analyst B
"Active User = user with a session"
Marketing
"Active User = user who watched content"
```
Now you have three numbers.
A proper Gold layer might establish:
```text
dim_user
fact_viewing
fact_sessions
fact_subscriptions
```
and governed KPI definitions determine how those datasets are interpreted.
We'll get into **fact and dimension tables** in another lesson.
---
# 5. Something important: Medallion ≠ Star Schema
You will eventually hear both terms.
They're related, but they're answering different questions.
**Medallion architecture asks:**
> How processed is the data?
```text
Bronze → Silver → Gold
```
**Star schema asks:**
> How should analytical data be modeled?
```text
Dimension
|
Dimension - Fact - Dimension
|
Dimension
```
A **Gold-layer dataset could be modeled as a star schema**.
That's why these concepts often appear in the same conversation.
---
# 6. Common mistake
A company sometimes says:
> "We have a Gold layer."
But what they really have is:
```text
Raw Data
↓
A bunch of SQL
↓
Power BI
```
The word "Gold" doesn't automatically mean the data is trustworthy.
A CTO should care about:
- Ownership
- Documentation
- Data quality
- Business definitions
- Lineage
- Testing
We'll cover **Data Lineage** separately because it's another term you're very likely to encounter.
---
# 7. Meeting scenario
Imagine your data analyst says:
> "We should create this KPI in the Gold layer."
You could respond:
> **"That makes sense. Before we do that, I want to make sure we agree on the business definition of the KPI and who owns that definition. Once that's established, we can expose the governed metric through the Gold layer so Power BI and other consumers use the same calculation."**
That's a strong manager/CTO answer.
You're not pretending to know how they'll implement it.
You're focusing on **governance, consistency and business value**.
---
# 8. Your cheat sheet
Remember only this:
> 🥉 **Bronze:** What did we receive?
>
> 🥈 **Silver:** What data is technically clean and trustworthy?
>
> 🥇 **Gold:** What does the business need?
>
> **Medallion architecture:** Raw → Clean → Business-ready
>
> **Your CTO question:** Who owns it, who defines it, and can we trust it?
---
## Add this to Obsidian
I'd create:
```text
Knowledge
└── CTO Academy
└── Data
└── Medallion Architecture.md
```
At the bottom:
```markdown
## Related Concepts
- [[Data Warehouse]]
- [[Data Lake vs Data Warehouse vs Lakehouse]]
- [[Lakehouse]]
- [[ETL]]
- [[ELT]]
- [[Star Schema]]
- [[Fact Table]]
- [[Dimension Table]]
- [[Data Lineage]]
- [[Data Governance]]
- [[Semantic Layer]]
```
### One question for you before we consider today's lesson complete:
Your Power BI team currently receives data from multiple sources and creates reports.
**Where do you think your current environment sits today?**
Do you think you have something resembling Bronze → Silver → Gold, or are your analysts mostly taking source data and doing the cleaning/business logic directly inside Power BI?
My answer:
We definitely have a bronze, silver and gold. Power Bi is only used to create the reports so it consumes the gold layer. I do know there's azure synapse, data lake, data warehouses, so all of these should be for bronze and silver layers.