amadou/SecondBrain
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

vault backup: 2026-07-19 16:34:17

Browse Source
This commit is contained in:
amadou
2026-07-19 16:34:17 +02:00
parent fda2206f43
commit d560da5af2
33 changed files with 3289 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
10 Knowledge/CTO Academy/Data/Semantic Layer.md
+637
View File
@@ -0,0 +1,637 @@
**Estimated reading time:** 10–12 minutes
# Semantic Layer
## Definition
A **Semantic Layer** is a business-friendly layer that sits between raw or modeled data and the tools people use to analyze it.
Its purpose is to translate technical data structures into consistent business concepts.
For example, instead of every analyst independently deciding how to calculate:
- Revenue
- Unique Viewer
- Active User
- Watch Time
- Conversion Rate
the Semantic Layer can define those metrics once and expose them consistently to tools like Power BI.
Think of it as:
> **The layer that gives business meaning to data.**
---
## Simple Mental Model
Think:
```text
Data Model
↓
Semantic Layer
↓
Business Meaning
```
Without a Semantic Layer, users may see:
```text
fact_viewing
dim_content
user_key
watch_seconds
session_id
```
With a Semantic Layer, they see:
```text
Total Watch Hours
Unique Viewers
Content Title
Platform
Monthly Audience
```
The underlying data may be the same.
The Semantic Layer makes it understandable and consistent.
---
## How It Works
Imagine your Gold layer contains:
```text
fact_viewing
dim_content
dim_platform
dim_date
```
This might be modeled as a [[Star Schema]].
The raw fields might include:
```text
watch_seconds
user_key
content_key
date_key
platform_key
```
The Semantic Layer can define:
```text
Total Watch Hours
= SUM(watch_seconds) / 3600
```
It can also define:
```text
Unique Viewers
= distinct count of user_key
```
And expose friendly dimensions such as:
```text
Content Title
Genre
Platform
Month
Year
```
The business user does not need to understand:
```text
JOIN fact_viewing
ON dim_content.content_key = fact_viewing.content_key
```
They simply use:
> Content Title
and:
> Total Watch Hours
That abstraction is the value of the Semantic Layer.
---
## Example
Suppose you have two analysts.
### Analyst A
Calculates:
```text
Unique Viewer
= DISTINCTCOUNT(user_id)
```
### Analyst B
Calculates:
```text
Unique Viewer
= DISTINCTCOUNT(session_id)
```
They both publish dashboards labeled:
> Unique Viewers
But the numbers are different.
The problem is not necessarily Power BI.
The problem is that the business definition was not centralized.
A governed Semantic Layer could define:
```text
Unique Viewer
= DISTINCTCOUNT(user_id)
```
once.
Then every report uses the same metric.
This creates:
```text
One definition
↓
One metric
↓
Many reports
```
instead of:
```text
Report A
→ Definition A
Report B
→ Definition B
Report C
→ Definition C
```
---
## How It Fits Into the Bigger Picture
We can now connect everything you've learned so far:
```text
SOURCE SYSTEMS
GA4
JW Player
OTT
CRM
Finance
↓
[[ETL vs ELT]]
↓
[[Medallion Architecture]]
Bronze
Raw Data
↓
Silver
Clean Data
↓
Gold
Business-Ready Data
↓
[[Fact Table]]
+
[[Dimension Table]]
↓
[[Star Schema]]
or
[[Snowflake Schema]]
↓
Semantic Layer
↓
Power BI
↓
Business Users
```
The important distinction is:
> [[Star Schema]] and [[Snowflake Schema]] organize the analytical data.
> The Semantic Layer defines how the business understands and consumes that data.
---
## Semantic Layer vs Gold Layer
These concepts are related, but they are not the same.
The Gold layer is part of [[Medallion Architecture]].
It represents:
> Business-ready data.
The Semantic Layer represents:
> Business meaning and definitions exposed to consumers.
A Gold dataset might contain:
```text
fact_viewing
dim_content
dim_platform
```
The Semantic Layer might expose:
```text
Watch Hours
Unique Viewers
Average Watch Time
Platform
Genre
Month
```
So conceptually:
```text
Gold Layer
↓
Technical analytical model
↓
Semantic Layer
↓
Business-friendly metrics and dimensions
```
In some architectures, the boundary between Gold and the Semantic Layer can feel blurry.
That's okay.
The key is to understand the responsibility:
> Gold prepares trusted business-ready data.
> The Semantic Layer gives that data consistent business meaning.
---
## Semantic Layer vs Power BI Report
A Power BI report is the visualization and presentation layer.
The Semantic Layer should ideally sit underneath it.
Conceptually:
```text
Data
↓
Semantic Layer
↓
Power BI Dataset / Model
↓
Reports
↓
Dashboards
```
This matters because a report should not have to redefine the business from scratch.
For example, you don't want:
```text
Marketing Dashboard
→ defines Active User
Executive Dashboard
→ defines Active User differently
Product Dashboard
→ another definition
```
You want:
```text
Semantic Layer
↓
Active User
↓
Marketing Dashboard
Executive Dashboard
Product Dashboard
```
One definition.
Many consumers.
---
## Business Metrics
One of the most important roles of a Semantic Layer is metric governance.
Consider:
```text
Revenue
```
Sounds simple.
But does Revenue mean:
- Gross revenue?
- Net revenue?
- Before tax?
- After refunds?
- After discounts?
- Recognized revenue?
- Invoiced revenue?
The word itself is not enough.
The Semantic Layer should connect the label:
```text
Revenue
```
to a governed business definition.
The same applies to:
```text
Audience
Unique Viewer
Subscriber
Active User
Conversion
Engagement
Completion Rate
```
Without governance, these terms can mean different things to different teams.
---
## My Company / Real-World Context
For your environment, I can imagine metrics such as:
```text
Total Watch Hours
Unique Viewers
Average Watch Time
Video Starts
Completion Rate
Sessions
Active Users
Content Reach
Ad Impressions
```
The problem is that many of these are open to interpretation.
For example:
> What is a Unique Viewer?
Is it:
```text
Logged-in user?
Device?
Cookie?
Anonymous browser?
Household?
Account?
```
Or:
> What is Completion Rate?
Is it:
```text
Watched 90%?
Watched 95%?
Reached the final second?
```
A Semantic Layer is where these definitions should become standardized.
This is directly related to your work in [[Data Governance]].
The business should not depend on each analyst remembering the correct interpretation.
The definition should exist in the architecture.
---
## CTO Perspective
A CTO should care deeply about the Semantic Layer because it connects technical systems to business decisions.
The main risk is not:
> "The dashboard is ugly."
The main risk is:
> "The dashboard is confidently showing the wrong business meaning."
A good Semantic Layer helps create:
- Consistent KPIs
- Reusable metrics
- Shared definitions
- Fewer reporting disputes
- Better trust in analytics
- Faster report development
It also reduces dependency on individual analysts.
If Analyst A leaves the company, the definition of a KPI should not disappear with them.
The knowledge should exist in the system.
---
### Questions to Ask
- Where are our KPI definitions stored?
- Who owns the business definition of each KPI?
- Are metrics defined once or independently in each report?
- Can multiple reports reuse the same governed metric?
- Are business users seeing friendly names instead of technical fields?
- How do we handle changes to KPI definitions?
- How are metric definitions documented?
- Can we trace a metric through [[Data Lineage]] to its source?
- Are analysts allowed to create their own competing definitions?
- Which team owns the Semantic Layer?
---
## Meeting Scenario
**Situation:**
Marketing says:
> "Our active users increased by 12%."
Product says:
> "No, active users only increased by 5%."
Both teams are using Power BI.
The discussion starts focusing on:
> "Which dashboard is correct?"
**Possible response:**
> "Before comparing the dashboards, can we confirm whether both teams are using the same governed definition of Active User? If the metric is defined differently in each report, then the issue is not the visualization. It's that we don't have one shared business definition."
If they confirm the definitions are different:
> "Then I think we should centralize the definition in the Semantic Layer and have both reports consume the same metric. Otherwise, we'll keep debating numbers instead of making decisions."
This reframes the problem from:
> Which analyst made the mistake?
to:
> Why does the architecture allow multiple definitions of the same KPI?
That is a much more strategic question.
---
## Key Takeaways
- A **Semantic Layer** translates technical data structures into consistent business concepts.
- It sits between analytical data models and reporting tools.
- It defines reusable metrics and business-friendly dimensions.
- It helps ensure that the same KPI means the same thing everywhere.
- [[Star Schema]] and [[Snowflake Schema]] organize data.
- The Semantic Layer gives that data business meaning.
- The Semantic Layer and Gold layer are related but not identical.
- Power BI reports should ideally consume governed metrics instead of redefining business logic independently.
- A strong Semantic Layer is a major part of [[Data Governance]].
## Related Concepts
- [[Data Governance]]
- [[Data Lineage]]
- [[Star Schema]]
- [[Snowflake Schema]]
- [[Fact Table]]
- [[Dimension Table]]
- [[Measure]]
- [[Medallion Architecture]]
- [[Data Warehouse]]
- [[KPI]]
- [[Metric]]
- [[Business Glossary]]
The next lesson I recommend is **Lesson 9: `Data Lineage.md`**. It naturally follows this because once metrics and business definitions are governed, the next question is: **"Where did this number come from?"**