amadou/SecondBrain
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

vault backup: 2026-07-19 16:34:17

Browse Source
This commit is contained in:
amadou
2026-07-19 16:34:17 +02:00
parent fda2206f43
commit d560da5af2
33 changed files with 3289 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
10 Knowledge/CTO Academy/Data/.gitkeep
View File
10 Knowledge/CTO Academy/Data/Dimension Table.md
+419
View File
@@ -0,0 +1,419 @@
## Definition
A **Dimension Table** contains descriptive information that provides **context** for the events and measurements stored in a [[Fact Table]].
If a Fact Table tells us:
> **What happened?**
Dimension Tables help answer:
> **Who? What? Where? When? How?**
Examples include:
- Content
- Customer
- User
- Product
- Platform
- Device
- Geography
- Date
---
## Simple Mental Model
Think:
> **Fact = What happened?**
> **Dimension = Describe what happened.**
If the Fact Table says:
> A viewing event lasted 125 seconds.
Dimensions tell us:
> **Who:** User 7821
> **What:** Episode 1 of Show A
> **When:** July 19, 2026
> **Where/How:** Roku
Together, Facts and Dimensions turn an event into something the business can analyze.
---
## How It Works
Imagine a [[Fact Table]] called:
`fact_viewing`
It contains:
|content_id|platform_id|date_id|watch_seconds|
|---|---|---|---|
|501|3|20260719|125|
The Fact Table knows:
`content_id = 501`
But `501` doesn't mean anything useful to a business user.
A Dimension Table called:
`dim_content`
could contain:
|content_id|title|series|genre|language|
|---|---|---|---|---|
|501|Episode 1|Show A|Drama|French|
Now we know what Content 501 represents.
Instead of storing:
`Episode 1 / Show A / Drama / French`
inside every viewing event, the Fact Table stores the key:
`501`
and connects it to `dim_content`.
---
## Example
Consider another Dimension Table:
`dim_platform`
|platform_id|platform|device_type|
|---|---|---|
|1|Web|Desktop|
|2|iOS|Mobile|
|3|Roku|OTT|
|4|Android TV|OTT|
The [[Fact Table]] contains:
`platform_id = 3`
The Dimension Table tells us:
`3 = Roku`
Now the business can ask:
> How many hours were watched on Roku?
Or:
> How does OTT viewing compare with mobile viewing?
The Fact Table provides the measurable event.
The Dimension Table provides the context needed to group and analyze those events.
---
## The Date Dimension
One of the most common Dimension Tables is:
`dim_date`
It might contain:
|date_id|date|day|month|quarter|year|weekend|
|---|---|---|---|---|---|---|
|20260719|2026-07-19|Sunday|July|Q3|2026|Yes|
You might wonder why an entire table is needed just for dates.
A Date Dimension makes it easier to analyze data by:
- Day
- Week
- Month
- Quarter
- Year
- Weekend vs weekday
Organizations can also add business-specific calendars:
- Fiscal year
- Fiscal quarter
- Broadcast season
- Programming season
This allows consistent time-based analysis across multiple [[Fact Table|Fact Tables]].
---
## Natural Keys and Surrogate Keys
Dimension Tables commonly introduce the distinction between a [[Natural Key]] and a [[Surrogate Key]].
A Natural Key comes from the business or source system.
Examples:
- Employee number
- Product SKU
- ISBN
- External content ID
A Surrogate Key is generated internally by the data platform.
For example:
`content_key = 501`
The number `501` may have no business meaning.
A Dimension Table could contain:
|content_key|jwplayer_id|title|
|---|---|---|
|501|abc123|Show A|
Here:
`content_key = 501`
is the [[Surrogate Key]].
`jwplayer_id = abc123`
is the source identifier and could potentially act as a [[Natural Key]].
Surrogate Keys are useful because source systems can change.
For example, a company might use:
JW Player → New Video Platform
The analytical data model can maintain its own internal identifiers rather than making its entire structure dependent on identifiers controlled by an external vendor.
This concept becomes particularly important when studying [[Slowly Changing Dimensions]].
---
## How It Fits Into the Bigger Picture
A simplified architecture could look like:
Sources
↓
[[ETL vs ELT]]
↓
[[Medallion Architecture]]
↓
Bronze
↓
Silver
↓
Gold
↓
[[Fact Table]] + Dimension Tables
↓
[[Star Schema]]
↓
[[Semantic Layer]]
↓
Power BI
Dimension Tables make analytical data easier for humans and BI tools to understand.
Instead of analyzing meaningless IDs, users can analyze:
- Show title
- Genre
- Platform
- Device
- Country
- Month
This is one of the ways raw technical data becomes **business-ready information**.
---
## My Company / Real-World Context
For a media company, useful Dimension Tables might include:
`dim_content`
Describes:
- Program
- Episode
- Series
- Genre
- Language
`dim_platform`
Describes:
- Web
- Mobile
- Roku
- Apple TV
- Android TV
`dim_device`
Describes:
- Desktop
- Mobile
- Tablet
- Connected TV
`dim_date`
Describes:
- Day
- Month
- Quarter
- Year
- Broadcast season
These dimensions could connect to a [[Fact Table]] such as:
`fact_viewing`
allowing Power BI users to analyze:
> Watch hours **by platform**
> Unique viewers **by content**
> Viewing sessions **by month**
> Engagement **by device**
The Fact Table provides the number.
The Dimension Table provides the **"by what?"**
---
## CTO Perspective
Dimension Tables may look like a technical modeling detail, but they have important governance implications.
Imagine one system says:
`French`
another says:
`FR`
and another says:
`fr_CA`
If every Power BI report handles these independently, the organization can end up with inconsistent results.
A governed Dimension Table can establish a standardized representation.
The same applies to:
- Content categories
- Platforms
- Departments
- Regions
- Products
- Customer segments
Dimension Tables can therefore become an important part of [[Data Governance]] because they help establish shared business definitions and classifications.
A CTO should care less about personally designing `dim_platform` and more about ensuring the organization has **consistent definitions across systems and reports**.
---
### Questions to Ask
- What business entity does this Dimension Table represent?
- What attributes describe it?
- What is its [[Natural Key]]?
- Does it use a [[Surrogate Key]]?
- Which source systems feed this dimension?
- How are conflicting source values standardized?
- Who owns the business definition?
- How are historical changes handled?
- Is this dimension shared across multiple [[Fact Table|Fact Tables]]?
- Are Power BI reports using the same governed dimensions?
---
## Meeting Scenario
**Situation:**
Two Power BI reports show different viewing totals by platform.
One report categorizes:
`Roku → Connected TV`
while another categorizes:
`Roku → OTT`
The teams are debating which report is correct.
**Possible response:**
> "It sounds like the issue isn't necessarily with the viewing data itself, but with how we're classifying platforms. Before changing either report, can we confirm whether we have a governed platform dimension and agree on the business classification we want to use?"
You could then add:
> "Once that definition is established centrally, both reports should consume the same classification rather than maintaining separate platform mappings."
This moves the discussion from:
> Which analyst has the correct Power BI report?
to:
> What is our organization's governed definition?
That's a [[Data Governance]] problem rather than simply a reporting problem.
---
## Key Takeaways
- A **Dimension Table** provides descriptive context for a [[Fact Table]].
- Facts answer **what happened**; dimensions answer **who, what, where, when, and how**.
- Dimensions allow data to be grouped and analyzed using business-friendly attributes.
- [[Natural Key|Natural Keys]] usually originate from business or source systems.
- [[Surrogate Key|Surrogate Keys]] are internally generated identifiers.
- Governed dimensions help create consistent definitions across reports and systems.
- Fact Tables and Dimension Tables commonly come together in a [[Star Schema]].
## Related Concepts
- [[Fact Table]]
- [[Grain]]
- [[Measure]]
- [[Natural Key]]
- [[Surrogate Key]]
- [[Slowly Changing Dimensions]]
- [[Star Schema]]
- [[Medallion Architecture]]
- [[Data Warehouse]]
- [[Data Governance]]
- [[Semantic Layer]]
10 Knowledge/CTO Academy/Data/Fact Table.md
+356
View File
@@ -0,0 +1,356 @@
## Definition
A **Fact Table** is a table in an analytical data model that records **business events, transactions, or measurable observations**.
Examples of business events include:
- A user watching a video
- A customer making a purchase
- An advertisement being displayed
- A website session occurring
- A subscription being created
Fact tables typically contain:
1. **Keys** that connect the event to descriptive information stored in [[Dimension Table|Dimension Tables]].
2. **Measures** representing numerical values that can be analyzed.
---
## Simple Mental Model
Think:
> **Fact Table = What happened, and how much?**
For a media company:
> Someone watched something, somewhere, at a certain time, for 125 seconds.
The **event** is the viewing.
The **125 seconds** is a measure.
The information describing who, what, when, and where comes from [[Dimension Table|Dimension Tables]].
---
## How It Works
Imagine a table called:
`fact_viewing`
|viewing_id|content_id|user_id|platform_id|date_id|watch_seconds|
|---|---|---|---|---|---|
|1001|501|7821|3|20260719|125|
|1002|502|9921|1|20260719|1800|
|1003|501|8812|2|20260719|340|
Each row represents something that happened.
For example:
> User 7821 watched Content 501 on Platform 3 for 125 seconds.
The columns:
- `content_id`
- `user_id`
- `platform_id`
- `date_id`
connect the event to [[Dimension Table|Dimension Tables]].
The column:
`watch_seconds`
is a [[Measure]].
Measures are values that can often be aggregated.
Examples include:
- Watch seconds
- Revenue
- Quantity
- Cost
- Clicks
- Impressions
- Number of sessions
For example:
`SUM(watch_seconds)`
could be used to calculate total watch time.
---
## Grain
One of the most important properties of a Fact Table is its [[Grain]].
The grain, or **granularité**, answers:
> **What does ONE ROW represent?**
Before designing a Fact Table, its grain should be clearly defined.
For example, `fact_viewing` could have the grain:
> One row represents one viewing session for one piece of content by one user.
If a user watches the same episode three separate times:
`3 viewing sessions = 3 rows`
But another Fact Table could have a different grain:
> One row represents one user/content/day combination.
In that model, watching the same episode three times on the same day might produce:
`1 row`
with:
`session_count = 3`
The grain determines which calculations are valid.
For this reason, one of the most useful questions when examining analytical data is:
> **"What does one row represent?"**
---
## Example
Imagine a report states:
> Total audience: 1,200,000
The underlying Fact Table has this grain:
> One row = one viewing session.
If the calculation is:
`COUNT(viewing_id)`
then the result is actually counting **viewing sessions**, not necessarily viewers.
One person could watch five times:
`5 viewing sessions`
but still represent:
`1 unique viewer`
Depending on the data model, calculating unique viewers might instead require something such as:
`COUNT(DISTINCT user_id)`
Understanding the [[Grain]] helps determine whether a metric actually represents what its label claims.
---
## Types of Fact Tables
Not every Fact Table represents an individual transaction.
Three common types are:
### Transaction Fact Table
One row represents an individual event.
Examples:
- One purchase
- One viewing session
- One ad impression
### Periodic Snapshot Fact Table
One row represents the state of something at a regular interval.
Examples:
- Daily subscriber count
- Monthly account balance
- Weekly inventory level
### Accumulating Snapshot Fact Table
One row tracks a process as it progresses through multiple stages.
For example:
Order placed → Order processed → Order shipped → Order delivered
The same row can be updated as the process progresses.
---
## How It Fits Into the Bigger Picture
As we learned in [[Medallion Architecture]], data can progressively move from raw to business-ready:
Bronze → Silver → Gold
A simplified media data architecture could look like:
Source systems
↓
[[ETL vs ELT]]
↓
Bronze: Raw viewing events
↓
Silver: Clean and standardized viewing events
↓
Gold: Analytical models
↓
`fact_viewing`
↓
Power BI
A Fact Table can therefore be part of the business-ready data model exposed to analytics tools.
The Fact Table does not normally exist alone.
It connects to [[Dimension Table|Dimension Tables]], creating analytical models such as a [[Star Schema]].
---
## My Company / Real-World Context
In a media company, potential Fact Tables could include:
- `fact_viewing`
- `fact_web_sessions`
- `fact_ad_impressions`
- `fact_subscriptions`
- `fact_linear_audience`
For `fact_viewing`, the measures might include:
- `watch_seconds`
- `sessions`
- `starts`
- `completions`
The Fact Table could connect to dimensions describing:
- Content
- Users
- Platforms
- Devices
- Dates
This could allow Power BI to answer questions such as:
> How many hours were watched last month?
> Which content generated the most viewing?
> Which platforms have the highest engagement?
> How has viewing changed over time?
The accuracy of those answers depends heavily on the [[Grain]] and the definition of the underlying [[Measure|Measures]].
---
## CTO Perspective
A CTO does not necessarily need to personally design every Fact Table.
However, when reviewing data or analytics, understanding the underlying Fact Table helps determine whether the reported metric is meaningful.
A particularly powerful question is:
> **"What's the grain of the underlying Fact Table?"**
Or, in simpler language:
> **"What exactly does one row represent?"**
This helps identify situations where:
- Sessions are presented as users
- Transactions are presented as customers
- Views are presented as audience
- Aggregated data is incorrectly aggregated again
A technology leader should also understand where the business logic behind important metrics is defined and whether different analysts are working from the same governed model.
This connects directly to [[Data Governance]].
---
### Questions to Ask
- What does one row in this Fact Table represent?
- What is the [[Grain]]?
- What are the primary measures?
- Which [[Dimension Table|Dimension Tables]] does it connect to?
- Where is the business logic defined?
- Can this measure safely be summed?
- Are we counting events or unique entities?
- Can we trace this metric back through [[Data Lineage]] to the original source?
- Are all analysts using the same governed definition?
---
## Meeting Scenario
**Situation:**
An analyst presents:
> "Our total audience increased by 25% this month."
You know the underlying data comes from viewing events, but you don't know exactly how "audience" was calculated.
**Possible response:**
> "Before we interpret the increase, can you clarify how we're defining audience and what the grain of the underlying Fact Table is? Are we measuring unique viewers, viewing sessions, or total views?"
The analyst responds:
> "We're counting viewing sessions."
You could respond:
> "Then I think we should be careful calling this audience growth. What we can confidently say is that viewing sessions increased by 25%. If we want to measure audience growth, we should use the appropriate unique-viewer definition."
This changes the conversation from challenging the analyst to clarifying the **definition and interpretation of the metric**.
---
## Key Takeaways
- A **Fact Table** records business events or measurable observations.
- A [[Measure]] is a numeric value being analyzed.
- [[Grain]] defines exactly what one row represents.
- Understanding the grain is essential before interpreting metrics.
- Fact Tables connect to [[Dimension Table|Dimension Tables]] to provide context.
- Fact Tables and Dimension Tables commonly form a [[Star Schema]].
- A CTO should always understand what is actually being counted before accepting a KPI.
## Related Concepts
- [[Dimension Table]]
- [[Grain]]
- [[Measure]]
- [[Star Schema]]
- [[Medallion Architecture]]
- [[ETL vs ELT]]
- [[Data Warehouse]]
- [[Data Governance]]
- [[Data Lineage]]
- [[Semantic Layer]]
10 Knowledge/CTO Academy/Data/Medallion Architecture.md
+1 -2
View File
@@ -382,8 +382,7 @@ At the bottom:
- [[Data Warehouse]]
- [[Data Lake vs Data Warehouse vs Lakehouse]]
- [[Lakehouse]]
- [[ETL]]
- [[ELT]]
- [[ETL vs ELT]]
- [[Star Schema]]
- [[Fact Table]]
- [[Dimension Table]]
10 Knowledge/CTO Academy/Data/Semantic Layer.md
+637
View File
@@ -0,0 +1,637 @@
**Estimated reading time:** 10–12 minutes
# Semantic Layer
## Definition
A **Semantic Layer** is a business-friendly layer that sits between raw or modeled data and the tools people use to analyze it.
Its purpose is to translate technical data structures into consistent business concepts.
For example, instead of every analyst independently deciding how to calculate:
- Revenue
- Unique Viewer
- Active User
- Watch Time
- Conversion Rate
the Semantic Layer can define those metrics once and expose them consistently to tools like Power BI.
Think of it as:
> **The layer that gives business meaning to data.**
---
## Simple Mental Model
Think:
```text
Data Model
↓
Semantic Layer
↓
Business Meaning
```
Without a Semantic Layer, users may see:
```text
fact_viewing
dim_content
user_key
watch_seconds
session_id
```
With a Semantic Layer, they see:
```text
Total Watch Hours
Unique Viewers
Content Title
Platform
Monthly Audience
```
The underlying data may be the same.
The Semantic Layer makes it understandable and consistent.
---
## How It Works
Imagine your Gold layer contains:
```text
fact_viewing
dim_content
dim_platform
dim_date
```
This might be modeled as a [[Star Schema]].
The raw fields might include:
```text
watch_seconds
user_key
content_key
date_key
platform_key
```
The Semantic Layer can define:
```text
Total Watch Hours
= SUM(watch_seconds) / 3600
```
It can also define:
```text
Unique Viewers
= distinct count of user_key
```
And expose friendly dimensions such as:
```text
Content Title
Genre
Platform
Month
Year
```
The business user does not need to understand:
```text
JOIN fact_viewing
ON dim_content.content_key = fact_viewing.content_key
```
They simply use:
> Content Title
and:
> Total Watch Hours
That abstraction is the value of the Semantic Layer.
---
## Example
Suppose you have two analysts.
### Analyst A
Calculates:
```text
Unique Viewer
= DISTINCTCOUNT(user_id)
```
### Analyst B
Calculates:
```text
Unique Viewer
= DISTINCTCOUNT(session_id)
```
They both publish dashboards labeled:
> Unique Viewers
But the numbers are different.
The problem is not necessarily Power BI.
The problem is that the business definition was not centralized.
A governed Semantic Layer could define:
```text
Unique Viewer
= DISTINCTCOUNT(user_id)
```
once.
Then every report uses the same metric.
This creates:
```text
One definition
↓
One metric
↓
Many reports
```
instead of:
```text
Report A
→ Definition A
Report B
→ Definition B
Report C
→ Definition C
```
---
## How It Fits Into the Bigger Picture
We can now connect everything you've learned so far:
```text
SOURCE SYSTEMS
GA4
JW Player
OTT
CRM
Finance
↓
[[ETL vs ELT]]
↓
[[Medallion Architecture]]
Bronze
Raw Data
↓
Silver
Clean Data
↓
Gold
Business-Ready Data
↓
[[Fact Table]]
+
[[Dimension Table]]
↓
[[Star Schema]]
or
[[Snowflake Schema]]
↓
Semantic Layer
↓
Power BI
↓
Business Users
```
The important distinction is:
> [[Star Schema]] and [[Snowflake Schema]] organize the analytical data.
> The Semantic Layer defines how the business understands and consumes that data.
---
## Semantic Layer vs Gold Layer
These concepts are related, but they are not the same.
The Gold layer is part of [[Medallion Architecture]].
It represents:
> Business-ready data.
The Semantic Layer represents:
> Business meaning and definitions exposed to consumers.
A Gold dataset might contain:
```text
fact_viewing
dim_content
dim_platform
```
The Semantic Layer might expose:
```text
Watch Hours
Unique Viewers
Average Watch Time
Platform
Genre
Month
```
So conceptually:
```text
Gold Layer
↓
Technical analytical model
↓
Semantic Layer
↓
Business-friendly metrics and dimensions
```
In some architectures, the boundary between Gold and the Semantic Layer can feel blurry.
That's okay.
The key is to understand the responsibility:
> Gold prepares trusted business-ready data.
> The Semantic Layer gives that data consistent business meaning.
---
## Semantic Layer vs Power BI Report
A Power BI report is the visualization and presentation layer.
The Semantic Layer should ideally sit underneath it.
Conceptually:
```text
Data
↓
Semantic Layer
↓
Power BI Dataset / Model
↓
Reports
↓
Dashboards
```
This matters because a report should not have to redefine the business from scratch.
For example, you don't want:
```text
Marketing Dashboard
→ defines Active User
Executive Dashboard
→ defines Active User differently
Product Dashboard
→ another definition
```
You want:
```text
Semantic Layer
↓
Active User
↓
Marketing Dashboard
Executive Dashboard
Product Dashboard
```
One definition.
Many consumers.
---
## Business Metrics
One of the most important roles of a Semantic Layer is metric governance.
Consider:
```text
Revenue
```
Sounds simple.
But does Revenue mean:
- Gross revenue?
- Net revenue?
- Before tax?
- After refunds?
- After discounts?
- Recognized revenue?
- Invoiced revenue?
The word itself is not enough.
The Semantic Layer should connect the label:
```text
Revenue
```
to a governed business definition.
The same applies to:
```text
Audience
Unique Viewer
Subscriber
Active User
Conversion
Engagement
Completion Rate
```
Without governance, these terms can mean different things to different teams.
---
## My Company / Real-World Context
For your environment, I can imagine metrics such as:
```text
Total Watch Hours
Unique Viewers
Average Watch Time
Video Starts
Completion Rate
Sessions
Active Users
Content Reach
Ad Impressions
```
The problem is that many of these are open to interpretation.
For example:
> What is a Unique Viewer?
Is it:
```text
Logged-in user?
Device?
Cookie?
Anonymous browser?
Household?
Account?
```
Or:
> What is Completion Rate?
Is it:
```text
Watched 90%?
Watched 95%?
Reached the final second?
```
A Semantic Layer is where these definitions should become standardized.
This is directly related to your work in [[Data Governance]].
The business should not depend on each analyst remembering the correct interpretation.
The definition should exist in the architecture.
---
## CTO Perspective
A CTO should care deeply about the Semantic Layer because it connects technical systems to business decisions.
The main risk is not:
> "The dashboard is ugly."
The main risk is:
> "The dashboard is confidently showing the wrong business meaning."
A good Semantic Layer helps create:
- Consistent KPIs
- Reusable metrics
- Shared definitions
- Fewer reporting disputes
- Better trust in analytics
- Faster report development
It also reduces dependency on individual analysts.
If Analyst A leaves the company, the definition of a KPI should not disappear with them.
The knowledge should exist in the system.
---
### Questions to Ask
- Where are our KPI definitions stored?
- Who owns the business definition of each KPI?
- Are metrics defined once or independently in each report?
- Can multiple reports reuse the same governed metric?
- Are business users seeing friendly names instead of technical fields?
- How do we handle changes to KPI definitions?
- How are metric definitions documented?
- Can we trace a metric through [[Data Lineage]] to its source?
- Are analysts allowed to create their own competing definitions?
- Which team owns the Semantic Layer?
---
## Meeting Scenario
**Situation:**
Marketing says:
> "Our active users increased by 12%."
Product says:
> "No, active users only increased by 5%."
Both teams are using Power BI.
The discussion starts focusing on:
> "Which dashboard is correct?"
**Possible response:**
> "Before comparing the dashboards, can we confirm whether both teams are using the same governed definition of Active User? If the metric is defined differently in each report, then the issue is not the visualization. It's that we don't have one shared business definition."
If they confirm the definitions are different:
> "Then I think we should centralize the definition in the Semantic Layer and have both reports consume the same metric. Otherwise, we'll keep debating numbers instead of making decisions."
This reframes the problem from:
> Which analyst made the mistake?
to:
> Why does the architecture allow multiple definitions of the same KPI?
That is a much more strategic question.
---
## Key Takeaways
- A **Semantic Layer** translates technical data structures into consistent business concepts.
- It sits between analytical data models and reporting tools.
- It defines reusable metrics and business-friendly dimensions.
- It helps ensure that the same KPI means the same thing everywhere.
- [[Star Schema]] and [[Snowflake Schema]] organize data.
- The Semantic Layer gives that data business meaning.
- The Semantic Layer and Gold layer are related but not identical.
- Power BI reports should ideally consume governed metrics instead of redefining business logic independently.
- A strong Semantic Layer is a major part of [[Data Governance]].
## Related Concepts
- [[Data Governance]]
- [[Data Lineage]]
- [[Star Schema]]
- [[Snowflake Schema]]
- [[Fact Table]]
- [[Dimension Table]]
- [[Measure]]
- [[Medallion Architecture]]
- [[Data Warehouse]]
- [[KPI]]
- [[Metric]]
- [[Business Glossary]]
The next lesson I recommend is **Lesson 9: `Data Lineage.md`**. It naturally follows this because once metrics and business definitions are governed, the next question is: **"Where did this number come from?"**
10 Knowledge/CTO Academy/Data/Snowflake Schema.md
+694
View File
@@ -0,0 +1,694 @@
**Estimated reading time:** 10–12 minutes
## Definition
A **Snowflake Schema** is an analytical data model where a central [[Fact Table]] connects to [[Dimension Table|Dimension Tables]], but some of those dimensions are further divided into additional related tables.
It is similar to a [[Star Schema]], but the dimensions are more **normalized**, meaning descriptive information is separated into additional tables instead of being stored together in one larger dimension.
A simplified comparison:
```text
STAR SCHEMA
dim_date
│
│
dim_content ───── fact_viewing ───── dim_user
│
│
dim_platform
```
Versus:
```text
SNOWFLAKE SCHEMA
dim_genre
│
▼
dim_date ─── fact_viewing ─── dim_content ─── dim_series
│ │
│ ▼
│ dim_language
│
dim_platform
```
The branching dimensions create a shape resembling a snowflake.
---
## Simple Mental Model
Think:
> **Star Schema = Keep dimensions together.**
> **Snowflake Schema = Break dimensions into smaller related tables.**
For example, in a [[Star Schema]], `dim_content` might contain:
```text
Content
├── Title
├── Series
├── Genre
└── Language
```
In a Snowflake Schema, it might become:
```text
Content
├── Title
├── Series ID ──────→ dim_series
├── Genre ID ───────→ dim_genre
└── Language ID ────→ dim_language
```
The information is still available, but it is distributed across more tables.
---
## How It Works
Let's use the media example from [[Star Schema]].
In a Star Schema, we might have:
### `dim_content`
|content_key|title|series|genre|language|
|---|---|---|---|---|
|501|Episode 1|Show A|Drama|French|
|502|Episode 2|Show A|Drama|French|
|503|Episode 1|Show B|Documentary|English|
Notice the repetition:
```text
Show A
Drama
French
```
appears multiple times.
A Snowflake Schema could separate this information.
### `dim_content`
|content_key|title|series_key|genre_key|language_key|
|---|---|---|---|---|
|501|Episode 1|10|5|1|
|502|Episode 2|10|5|1|
|503|Episode 1|11|8|2|
### `dim_series`
|series_key|series_name|
|---|---|
|10|Show A|
|11|Show B|
### `dim_genre`
|genre_key|genre_name|
|---|---|
|5|Drama|
|8|Documentary|
### `dim_language`
|language_key|language_name|
|---|---|
|1|French|
|2|English|
Now the relationships look like:
```text
fact_viewing
│
▼
dim_content
│
├────→ dim_series
│
├────→ dim_genre
│
└────→ dim_language
```
Instead of storing `Drama` repeatedly in `dim_content`, the model stores:
```text
genre_key = 5
```
and `dim_genre` tells us:
```text
5 = Drama
```
---
## Example
Suppose the business wants to know:
> "How many hours of French Drama content were watched last month?"
In a [[Star Schema]], the query conceptually follows:
```text
fact_viewing
│
▼
dim_content
│
├── genre = Drama
└── language = French
```
In a Snowflake Schema, it follows more relationships:
```text
dim_genre
│
│ Drama
▼
fact_viewing ─── dim_content
▲
│ French
│
dim_language
```
Both architectures can answer the same business question.
The difference is **how the descriptive data is organized**.
---
## Star Schema vs Snowflake Schema
The main difference is how dimensions are structured.
||[[Star Schema]]|Snowflake Schema|
|---|---|---|
|Dimensions|Less normalized|More normalized|
|Number of tables|Fewer|More|
|Relationships|Simpler|More complex|
|Queries|Generally simpler|Can require more joins|
|Data duplication|More|Less|
|BI usability|Often easier|Potentially more complex|
|Maintenance|Simpler model|More structured separation|
Neither approach is automatically correct or incorrect.
The architecture should depend on the use case.
---
## Why Use a Snowflake Schema?
There are situations where separating dimensions makes sense.
Imagine your company has:
```text
50,000 pieces of content
```
but only:
```text
20 genres
```
Instead of storing:
```text
Drama
Drama
Drama
Drama
Drama
Drama
...
```
thousands of times, you can store:
```text
genre_key = 5
```
and maintain the definition once:
```text
dim_genre
5 = Drama
```
This reduces duplication.
It can also make certain data easier to maintain centrally.
For example, if the organization changes:
```text
Children
```
to:
```text
Kids & Family
```
the change could potentially be made in one place.
---
## Why Not Snowflake Everything?
Because reducing duplication comes with a cost:
> **Complexity.**
Compare:
```text
fact_viewing
↓
dim_content
```
with:
```text
fact_viewing
↓
dim_content
↓
dim_series
↓
dim_genre
```
More tables mean:
- More relationships
- More joins
- More complexity for analysts
- More opportunities for incorrect relationships
- Potentially more complicated BI models
This is particularly relevant when the primary consumer is Power BI.
A model that is theoretically elegant from a database-design perspective may not necessarily be the easiest analytical model for business intelligence.
The goal isn't:
> Create the most normalized architecture possible.
The goal is:
> Create an architecture that reliably supports the analytical needs of the business.
---
## Normalization
The concept behind Snowflake Schemas is closely related to [[Normalization]].
Normalization means organizing data to reduce duplication and improve consistency.
Imagine:
```text
Employee
Amadou | Technology | Toronto
Alice | Technology | Toronto
Bob | Technology | Toronto
```
The values:
```text
Technology
Toronto
```
are repeated.
A more normalized structure could separate them:
```text
Employee
↓
Department
↓
Location
```
Normalization is extremely common in operational databases.
However, analytical systems often deliberately accept some duplication to make queries simpler and faster.
This is one reason [[Star Schema]] is popular in analytics.
The Star Schema is typically more **denormalized**.
The Snowflake Schema is more **normalized**.
---
## How It Fits Into the Bigger Picture
Our complete mental model is continuing to grow:
```text
SOURCE SYSTEMS
GA4
JW Player
OTT
Apps
CRM
│
▼
[[ETL vs ELT]]
│
▼
[[Medallion Architecture]]
Bronze
Raw Data
↓
Silver
Clean Data
↓
Gold
Business-Ready Data
│
▼
Analytical Data Model
│
├──── [[Star Schema]]
│
└──── Snowflake Schema
│
▼
[[Fact Table]]
+
[[Dimension Table|Dimension Tables]]
│
▼
[[Semantic Layer]]
│
▼
Power BI
```
Remember that [[Medallion Architecture]] and dimensional modeling solve different problems.
Medallion Architecture answers:
> **How refined is the data?**
Star and Snowflake Schemas answer:
> **How is the analytical data organized?**
---
## My Company / Real-World Context
Imagine your company has a content hierarchy:
```text
Brand
↓
Series
↓
Season
↓
Episode
```
You could design a single dimension:
```text
dim_content
content_key
episode
season
series
brand
genre
language
```
This would be closer to a [[Star Schema]].
Or you could separate the hierarchy:
```text
fact_viewing
│
▼
dim_content
│
▼
dim_season
│
▼
dim_series
│
▼
dim_brand
```
This is closer to a Snowflake Schema.
The second approach reduces duplication and separates the entities more explicitly.
But now imagine a Power BI analyst wants:
> Viewing hours by brand.
The relationship path becomes:
```text
fact_viewing
↓
dim_content
↓
dim_season
↓
dim_series
↓
dim_brand
```
instead of simply:
```text
fact_viewing
↓
dim_content.brand
```
The correct architecture depends on your data, governance requirements, performance, and how the model is consumed.
---
## CTO Perspective
As a CTO, the important question isn't:
> "Should we always use Star or Snowflake?"
The better question is:
> **"Which model best supports our analytical requirements while remaining understandable, maintainable, and governed?"**
A highly normalized model can be technically elegant but unnecessarily complex for BI users.
A highly denormalized model can be easy to consume but may introduce duplication and maintenance challenges.
Architecture is about trade-offs.
This is a recurring theme you'll encounter as a technology leader:
```text
Simplicity
↕
Flexibility
↕
Performance
↕
Maintainability
↕
Cost
```
There is rarely a universally perfect architecture.
The goal is to choose the architecture that best supports the business requirements.
---
### Questions to Ask
- Why are we using a Snowflake Schema instead of a [[Star Schema]]?
- What problem does the additional normalization solve?
- Does the additional complexity provide meaningful business or technical value?
- How many joins are required for common analytical queries?
- Is the model easy for Power BI developers to understand?
- Are dimensions being separated because they represent genuinely reusable business entities?
- Could some dimensions be simplified without losing important functionality?
- How does this affect query performance?
- How does this affect maintainability?
- Are shared definitions governed consistently?
- Does the model support the questions the business actually needs to answer?
---
## Meeting Scenario
**Situation:**
A data architect proposes redesigning the analytical model.
They want to separate:
```text
dim_content
```
into:
```text
dim_episode
dim_season
dim_series
dim_genre
dim_language
dim_brand
```
The reason given is:
> "It's more normalized and therefore cleaner."
**Possible response:**
> "I understand the normalization benefit, but before we redesign the model, I'd like to understand what problem the additional separation solves for us. Does it improve governance, maintainability, or performance enough to justify the added complexity for our analysts and Power BI models?"
If the answer is primarily:
> "It's technically cleaner."
You could respond:
> "Then I'd like us to compare the operational benefit against the additional complexity. If our primary use case is analytics, simplicity for the consumers of the model should also be part of the architectural decision."
This doesn't mean you're rejecting the proposal.
You're asking the team to justify an architectural decision based on **business and operational outcomes**, rather than technical elegance alone.
---
## Key Takeaways
- A **Snowflake Schema** is similar to a [[Star Schema]], but dimensions are further divided into related tables.
- Snowflake Schemas are more normalized.
- Star Schemas are generally more denormalized.
- Snowflake Schemas reduce data duplication but introduce additional relationships and complexity.
- Star Schemas are often simpler for BI and analytical workloads.
- Neither architecture is universally better.
- The choice should depend on business requirements, maintainability, performance, governance, and usability.
- Technical elegance alone is not sufficient justification for architectural complexity.
- A CTO should understand the **trade-off between normalization and simplicity**.
## Related Concepts
- [[Star Schema]]
- [[Fact Table]]
- [[Dimension Table]]
- [[Normalization]]
- [[Grain]]
- [[Measure]]
- [[Conformed Dimension]]
- [[Medallion Architecture]]
- [[Data Warehouse]]
- [[Semantic Layer]]
- [[Data Governance]]
- [[Data Lineage]]
---
The next lesson I recommend is **Lesson 8: [[Semantic Layer]]** (`Semantic Layer.md`). This one should be especially relevant to your actual job because it connects everything we've built so far to **Power BI, governed KPIs, business definitions, and the problem of two analysts producing different answers to what appears to be the same question**.
10 Knowledge/CTO Academy/Data/Star Schema.md
+741
View File
@@ -0,0 +1,741 @@
## Definition
A **Star Schema** is a way of organizing analytical data where a central [[Fact Table]] connects directly to multiple [[Dimension Table|Dimension Tables]].
It is called a **Star Schema** because when visualized, the structure often looks like a star:
```text
dim_date
│
│
▼
dim_content ────── fact_viewing ────── dim_user
▲
│
│
dim_platform
```
The [[Fact Table]] sits at the center.
The [[Dimension Table|Dimension Tables]] surround it and provide context.
Star Schemas are commonly used in [[Data Warehouse|Data Warehouses]] and analytical systems because they make data easier to understand, query, and analyze.
---
## Simple Mental Model
Think:
> **Fact Table = What happened?**
> **Dimension Tables = Who? What? Where? When? How?**
> **Star Schema = How we organize them together.**
For example:
```text
WHEN?
[[dim_date]]
│
│
WHAT? ─────── WHAT HAPPENED? ─────── WHO?
dim_content fact_viewing dim_user
│
│
HOW?
dim_platform
```
The Fact Table contains the measurable business event.
The Dimension Tables describe the event.
---
## How It Works
Imagine a media company has a [[Fact Table]]:
`fact_viewing`
|content_key|user_key|platform_key|date_key|watch_seconds|
|---|---|---|---|--:|
|501|7821|3|20260719|125|
|502|9921|1|20260719|1800|
|501|8812|2|20260719|340|
Remember from [[Fact Table]] that the [[Grain]] might be:
> One row represents one viewing session for one piece of content by one user.
The Fact Table contains the event and its [[Measure|Measures]].
Around it are dimensions.
### `dim_content`
|content_key|title|series|genre|language|
|---|---|---|---|---|
|501|Episode 1|Show A|Drama|French|
|502|Episode 5|Show B|Documentary|English|
### `dim_platform`
|platform_key|platform|device_type|
|---|---|---|
|1|Web|Desktop|
|2|iOS|Mobile|
|3|Roku|Connected TV|
### `dim_date`
|date_key|date|month|quarter|year|
|---|---|---|---|---|
|20260719|2026-07-19|July|Q3|2026|
The relationships allow us to combine the measurable event with descriptive context.
Conceptually:
```text
fact_viewing.watch_seconds
+
dim_content.genre
+
dim_platform.platform
+
dim_date.month
```
Now we can answer:
> How many hours of Drama were watched on Roku in July?
The Fact Table provides:
`watch_seconds`
The dimensions provide:
`Drama`
`Roku`
`July`
---
## Why Is It Called a Star?
Because dimensions connect **directly to the Fact Table**.
```text
dim_date
│
│
dim_content ──── fact_viewing ──── dim_user
│
│
dim_platform
```
Notice that:
`dim_content`
does not need to connect through another table before reaching:
`fact_viewing`
The relationships radiate outward from the center.
This creates the star-like shape.
---
## Example
Suppose the CEO asks:
> "What are our top five genres by total watch hours on Connected TV this year?"
The model could use:
```text
[[Fact Table]]
fact_viewing
↓
SUM(watch_seconds)
[[Dimension Table]]
dim_content
↓
genre
[[Dimension Table]]
dim_platform
↓
device_type = Connected TV
[[Dimension Table]]
dim_date
↓
year = 2026
```
The query conceptually becomes:
> Sum watch seconds
> Group by genre
> Filter device type to Connected TV
> Filter year to 2026
The Star Schema makes these relationships predictable and easy for analytical tools to navigate.
---
## How It Fits Into the Bigger Picture
We can now connect nearly everything we've learned so far.
```text
SOURCE SYSTEMS
GA4
JW Player
OTT
Apps
CRM
│
▼
[[ETL vs ELT]]
│
▼
[[Medallion Architecture]]
Bronze
Raw data
↓
Silver
Clean and standardized data
↓
Gold
Business-ready analytical data
│
▼
[[Star Schema]]
│
┌────┴────┐
│ │
[[Fact Table]]
+
[[Dimension Table|Dimension Tables]]
│
▼
[[Semantic Layer]]
│
▼
Power BI
│
▼
Business Users
```
A Star Schema can therefore be one way of organizing business-ready analytical data.
It is important to remember:
> **Gold Layer does not automatically mean Star Schema.**
A Gold Layer can contain many types of business-ready datasets.
However, Star Schemas are commonly used for analytical workloads because they organize Facts and Dimensions in a way that works well for BI.
---
## Star Schema vs One Giant Table
You might wonder:
> Why not just put everything into one table?
For example:
|user|content|genre|platform|device|date|watch_seconds|
|---|---|---|---|---|---|--:|
|A|Show A|Drama|Roku|TV|July 19|125|
|B|Show A|Drama|Roku|TV|July 19|300|
|C|Show A|Drama|Roku|TV|July 19|500|
This can work for small datasets.
But imagine hundreds of millions of viewing events.
The values:
`Show A`
`Drama`
`Roku`
`Connected TV`
could be repeated millions of times.
With a Star Schema, the Fact Table stores keys:
```text
content_key = 501
platform_key = 3
```
The descriptive information exists in the dimensions.
This also provides centralized definitions.
If Platform 3 is classified as:
`Connected TV`
that classification can be maintained in `dim_platform` instead of being independently recreated across many datasets.
---
## Star Schema vs Snowflake Schema
You will often hear these two terms together.
A Star Schema might look like:
```text
dim_date
│
│
dim_content ──── fact_viewing ──── dim_user
│
│
dim_platform
```
The dimensions connect directly to the Fact Table.
A [[Snowflake Schema]] further normalizes some dimensions.
For example, instead of:
```text
fact_viewing
│
▼
dim_content
title
series
genre
language
```
you might have:
```text
fact_viewing
│
▼
dim_content
│
├──── dim_genre
│
├──── dim_series
│
└──── dim_language
```
The structure begins branching outward.
Visually:
```text
dim_genre
│
▼
dim_date ───── fact_viewing ───── dim_content ─── dim_series
│ │
│ ▼
│ dim_language
│
dim_platform
```
This resembles a snowflake rather than a simple star.
We'll cover [[Snowflake Schema]] separately.
For now, remember:
> **Star Schema = Dimensions connect directly to Facts.**
> **Snowflake Schema = Dimensions may be broken into additional related tables.**
---
## Star Schema vs Medallion Architecture
These two concepts are easy to confuse because both describe data architecture.
But they answer completely different questions.
[[Medallion Architecture]] asks:
> **How refined is the data?**
```text
Bronze
↓
Silver
↓
Gold
```
[[Star Schema]] asks:
> **How is analytical data modeled?**
```text
Dimensions
↓
Fact Table
↑
Dimensions
```
You could therefore have:
```text
Gold Layer
│
├── Star Schema A
│
├── Star Schema B
│
└── Other business-ready datasets
```
One describes **data maturity**.
The other describes **data modeling**.
---
## Multiple Fact Tables
A data environment will usually have more than one Fact Table.
For your company, you could potentially have:
```text
fact_viewing
fact_web_sessions
fact_ad_impressions
fact_subscriptions
```
These could share some dimensions.
For example:
```text
dim_date
/ \
/ \
▼ ▼
fact_viewing fact_ad_impressions
│ │
└──── dim_content ───┘
```
The same `dim_content` could potentially describe content across multiple business processes.
A shared dimension used consistently across multiple Fact Tables is sometimes called a **conformed dimension**.
This becomes powerful because the organization develops common definitions.
Instead of every system having its own interpretation of:
> Content
you establish a shared business representation.
This connects directly to [[Data Governance]].
---
## My Company / Real-World Context
A media company's viewing model could potentially look like:
```text
dim_date
│
│
▼
dim_content ──────── fact_viewing ──────── dim_user
▲
│
┌───────────┴───────────┐
│ │
dim_platform dim_device
```
The Fact Table could contain measures such as:
- Watch seconds
- Viewing sessions
- Starts
- Completions
The dimensions could allow analysis by:
- Content
- Series
- Genre
- Platform
- Device
- Date
- Audience attributes
Power BI could then answer:
> What content generated the most watch hours?
> How is viewing distributed across platforms?
> Which genres perform best on Connected TV?
> How has viewing changed month over month?
> Are mobile users consuming different content from OTT users?
The Star Schema provides the underlying analytical structure that makes these questions easier to answer consistently.
---
## CTO Perspective
As a CTO, you do not need to personally design every Star Schema.
You should understand whether your analytical architecture is creating **consistent, reusable models** or whether every report is independently rebuilding business logic.
A warning sign would be:
```text
Power BI Report A
↓
Custom SQL
↓
Custom definition of Platform
Power BI Report B
↓
Different SQL
↓
Different definition of Platform
Power BI Report C
↓
Another transformation
↓
Another definition of Platform
```
A better governed architecture might be:
```text
Governed Data Model
[[Star Schema]]
│
▼
[[Semantic Layer]]
/ | \
/ | \
▼ ▼ ▼
Report A Report B Report C
```
This allows multiple reports to consume consistent business definitions.
The CTO-level question isn't:
> "Did you use a Star Schema?"
It is:
> **"Are we building reusable, governed analytical models, or is each report independently defining the business?"**
That's the larger architectural issue.
---
### Questions to Ask
- What business process does this Star Schema represent?
- What is the central [[Fact Table]]?
- What is its [[Grain]]?
- What are the [[Measure|Measures]]?
- Which [[Dimension Table|Dimension Tables]] surround it?
- Are dimensions shared across multiple Fact Tables?
- Are business definitions consistent across dimensions?
- Where is business logic defined?
- Are Power BI reports consuming governed models?
- Can metrics be traced through [[Data Lineage]] to their source?
- Is the model optimized for the questions the business actually needs to answer?
---
## Meeting Scenario
**Situation:**
Your two data analysts have created separate Power BI reports.
Both reports analyze video performance.
One analyst calculates viewing by joining directly to JW Player data and creates their own platform classifications.
The other uses a different dataset and maintains a separate mapping.
The reports now disagree on viewing by platform.
The discussion starts focusing on:
> "Which Power BI report is correct?"
**Possible response:**
> "Before we troubleshoot the individual reports, I think we should look at the underlying data model. Are both reports consuming the same governed Fact Table and platform dimension, or are we defining those relationships independently in each report?"
If they're independent:
> "Then the issue may be architectural rather than a Power BI issue. We should establish the governed definition once in the analytical model and have both reports consume the same structure."
You have now reframed the problem:
```text
Wrong question:
Which Power BI report is right?
↓
Better question:
Why can two reports define
the same business concept differently?
```
That's the connection between [[Star Schema]], [[Semantic Layer]], and [[Data Governance]].
---
## Key Takeaways
- A **Star Schema** organizes analytical data around a central [[Fact Table]] and surrounding [[Dimension Table|Dimension Tables]].
- The Fact Table represents measurable business events.
- Dimensions provide descriptive context.
- [[Grain]] defines what one row in the Fact Table represents.
- Star Schemas make analytical data easier to understand and query.
- [[Medallion Architecture]] describes data refinement; Star Schema describes data modeling.
- A Gold Layer can contain Star Schemas, but Gold does not automatically mean Star Schema.
- Multiple Fact Tables can share dimensions.
- Shared, governed dimensions help maintain consistent business definitions.
- Star Schemas can support reusable analytical models instead of rebuilding logic independently in every Power BI report.
## Related Concepts
- [[Fact Table]]
- [[Dimension Table]]
- [[Grain]]
- [[Measure]]
- [[Snowflake Schema]]
- [[Medallion Architecture]]
- [[Data Warehouse]]
- [[Semantic Layer]]
- [[Data Governance]]
- [[Data Lineage]]
- [[Conformed Dimension]]
---
For the next lesson, I recommend **Lesson 7: [[Snowflake Schema]]**. It's the natural comparison while [[Star Schema]] is fresh in your mind. After that, I would move to **[[Semantic Layer]]**, because that will connect the entire architecture you've learned so far directly to Power BI and to your responsibility as the manager of the data team.