9.3 KiB
Time: ~10–15 minutes
Today's goal is that if someone says:
"The data is in the lake."
or:
"We're moving to a lakehouse."
or:
"That's handled by Synapse."
you immediately understand what category of thing they're talking about.
1. Start with the simplest mental model
Think of these as answering different questions:
| Concept | Think of it as |
|---|---|
| Data Lake | A place to store huge amounts of data |
| Data Warehouse | A structured system optimized for analytics |
| Lakehouse | An architecture combining characteristics of both |
| Azure Synapse | A Microsoft analytics platform providing tools to work with data |
The first three describe data architectures/storage approaches.
Synapse is a product/platform.
That's the first distinction to remember.
2. Data Lake 🌊
A Data Lake is designed to store large quantities of data in many forms.
You might put:
GA4 event exports
JW Player logs
JSON files
CSV files
Parquet files
Application logs
Images
Video metadata
IoT data
Database exports
into a Data Lake.
The important thing is that the data doesn't necessarily need to be modeled for business reporting before you store it.
Think:
Store first. Figure out how to use it later.
For your company, something like Azure Data Lake Storage could contain enormous quantities of raw and processed data.
You could potentially organize it:
Data Lake
├── bronze/
│ ├── ga4/
│ ├── jwplayer/
│ └── ott/
│
├── silver/
│ ├── audience/
│ └── content/
│
└── gold/
├── audience_metrics/
└── content_performance/
So remember our previous lesson:
A Data Lake can contain Bronze, Silver AND Gold.
The Lake is where/how data is stored.
The Medallion layers describe the maturity of the data.
3. Data Warehouse 🏢
A Data Warehouse is much more structured.
It's designed primarily for:
Analytics and reporting.
Instead of throwing everything into it, you intentionally organize data for analysis.
Imagine:
DIM_DATE
│
│
DIM_CONTENT ─── FACT_VIEWING ─── DIM_USER
│
│
DIM_PLATFORM
Now you can easily ask:
How many hours were watched...
by content...
by platform...
by month...
by audience?
That's what warehouses are excellent at.
You'll eventually hear terms such as:
-
Fact tables
-
Dimension tables
-
Star schemas
-
Snowflake schemas
Those are closely associated with analytical data modeling and warehouses.
We'll cover those soon.
4. Data Lake vs Data Warehouse
The traditional distinction looks roughly like this:
| Data Lake | Data Warehouse | |
|---|---|---|
| Data | Raw + processed | Primarily structured |
| Structure | Flexible | Highly organized |
| Cost | Generally cheaper storage | Generally more expensive compute/storage |
| Purpose | Many possible uses | Analytics/reporting |
| Users | Data engineers/scientists | Analysts/BI |
| Schema | Often applied later | Typically defined deliberately |
A simplified architecture could therefore be:
SOURCE SYSTEMS
│
▼
DATA LAKE
│
Bronze
│
Silver
│
▼
DATA WAREHOUSE
│
Gold
│
▼
Power BI
Again, that's one possible architecture, not a universal rule.
5. The problem
Companies started asking:
Why are we maintaining two completely separate worlds?
They had:
Data Lake
AND
Data Warehouse
The lake was flexible and inexpensive.
The warehouse was structured and excellent for analytics.
But moving data between them added:
-
complexity
-
duplication
-
pipelines
-
governance challenges
-
additional cost
This contributed to the rise of the:
Lakehouse 🏠🌊
The idea is:
Can we get warehouse-like capabilities directly on top of lake-style storage?
Conceptually:
LAKEHOUSE
┌────────────────────────────┐
│ │
│ Bronze │
│ ↓ │
│ Silver │
│ ↓ │
│ Gold │
│ │
│ Structured + Unstructured │
│ │
│ BI + ML + Data Science │
│ │
└────────────────────────────┘
│
▼
Power BI
Technologies such as Delta Lake helped make this architecture practical by adding database-like capabilities to data stored in a lake.
The big idea:
Data Lake flexibility + Data Warehouse capabilities = Lakehouse
That's simplified, but it's the mental model I want you to remember.
6. Where does Azure Synapse fit?
This is where people get confused.
Azure Synapse Analytics is not simply a Data Lake.
It's not simply a Data Warehouse either.
It's a Microsoft analytics platform.
It provides capabilities around things such as:
Data
↓
Pipelines
↓
Transformations
↓
SQL
↓
Spark
↓
Analytics
It can interact with services such as Azure Data Lake Storage and provide SQL and Spark-based analytics capabilities.
So someone saying:
"Our data is in Synapse."
is a little like saying:
"Our website is in AWS."
Okay...
But where and how?
The CTO question becomes:
"Can you show me how we're using Synapse in our architecture?"
Maybe they're using:
-
Synapse pipelines for data movement.
-
Spark for transformations.
-
Serverless SQL to query files in the Data Lake.
-
Dedicated SQL pools as a warehouse.
Those are very different things.
7. Connecting Lessons 1 and 2
Now we have two separate dimensions.
Data maturity
Bronze
↓
Silver
↓
Gold
Data architecture
Data Lake
Data Warehouse
Lakehouse
And then we have a product:
Azure Synapse
These concepts can overlap.
For example:
Azure Platform
│
▼
Azure Data Lake Storage
Bronze → Silver → Gold
│
▼
Synapse SQL
│
▼
Power BI
That's a perfectly plausible architecture.
But your actual company architecture could be different.
8. CTO perspective
Imagine you're in a meeting and someone says:
"We need a Lakehouse."
Don't immediately say yes.
Ask:
"What problem are we solving that our current Data Lake and Warehouse architecture can't solve?"
That's the executive question.
Because "Lakehouse" sounds modern.
That doesn't automatically make it necessary.
Then ask:
"Where does our Gold layer currently live?"
"Are we duplicating data between our Lake and Warehouse?"
"What's driving our current cost?"
"What workloads are we supporting: BI, data science, ML, or all three?"
"How is data governed across these environments?"
Now you're participating in an architecture discussion without needing to write a single Spark job.
9. Meeting scenario
A data architect tells you:
"We should migrate our architecture to a Lakehouse."
A weaker response would be:
"Okay, what's involved?"
A stronger CTO response:
"Before we discuss the technology, I'd like to understand the problem we're solving. Are we trying to reduce duplication between the Lake and Warehouse, improve performance, lower cost, or support new workloads? Once we agree on the problem, we can determine whether a Lakehouse is actually the right architecture."
Notice our executive communication training appearing again?
Control the structure.
You just changed:
Should we implement a Lakehouse?
into:
What problem are we trying to solve?
That's exactly the skill you told me you want to develop.
10. Your cheat sheet
Remember these four sentences:
🌊 Data Lake: Store large amounts of diverse data flexibly.
🏢 Data Warehouse: Organize structured data for analytics and reporting.
🏠 Lakehouse: Bring warehouse-like capabilities to lake-style storage.
🧰 Azure Synapse: Microsoft's analytics platform that provides multiple tools for working with and analyzing data.
And remember:
Bronze/Silver/Gold tells me how refined the data is.
Lake/Warehouse/Lakehouse tells me about the architecture.
Synapse tells me about the technology/platform being used.
That's the distinction I want you to walk away with tonight.
Obsidian
Create:
Knowledge
└── CTO Academy
└── Data
└── Data Lake vs Data Warehouse vs Lakehouse.md
At the bottom:
## Related Concepts
- [[Medallion Architecture]]
- [[Azure Synapse]]
- [[ETL vs ELT]]
- [[Star Schema]]
- [[Fact Table]]
- [[Dimension Table]]
- [[Data Lake]]
- [[Data Warehouse]]
- [[Lakehouse]]
Don't create individual pages for all of them yet.
Lesson 3 should be ETL vs ELT. That's the missing bridge between today's lesson and the first lesson: how does the data actually travel from the source through Bronze, Silver and Gold?