vault backup: 2026-07-23 07:56:59

This commit is contained in:
2026-07-23 07:56:59 -04:00
parent 1033a565cb
commit af0951d4db
11 changed files with 1355 additions and 0 deletions
@@ -0,0 +1,119 @@
**MD file:** `Business Glossary.md`
**Estimated reading time:** 911 minutes
# Business Glossary
## Definition
A **Business Glossary** is a governed collection of business terms and their agreed definitions.
Its purpose is to make sure important words mean the same thing across the organization.
## Simple Mental Model
> **A Business Glossary is the organization's dictionary.**
Each important term can include a definition, owner, status, exclusions, and related metrics.
## How It Works
Example:
```text
Term: Unique Viewer
Definition: A distinct viewer identity with at least one valid content viewing event during the selected period.
Owner: Digital Product
Exclusions: Internal traffic, automated traffic, invalid playback events
```
The technical implementation may live in [[Semantic Layer]], but the meaning is documented in the glossary.
## Example
Without a glossary:
```text
Marketing: Subscriber = email newsletter subscriber
Product: Subscriber = paid platform subscriber
Finance: Subscriber = billable customer
```
All can make sense in context. The problem is when a dashboard simply says "Subscribers."
## How It Fits Into the Bigger Picture
```text
[[Data Owner]]
Approves business meaning
[[Business Glossary]]
Documents meaning
[[Semantic Layer]]
Implements governed metrics
```
## My Company / Real-World Context
Useful governed terms could include:
- Session
- Viewer
- Unique Viewer
- Audience
- Video Start
- Completion Rate
- Watch Hour
- Active User
- Subscriber
- Registered User
- OTT
- Connected TV
## CTO Perspective
Many reporting disagreements are vocabulary disagreements disguised as technical problems.
A CTO can often improve the discussion by asking:
> **Are we actually using the same definition?**
Start with terms tied to executive decisions, KPIs, financial impact, or repeated disputes.
### Questions to Ask
- Which terms repeatedly cause confusion?
- Does each critical term have an agreed definition?
- Who owns it?
- Is it connected to [[Semantic Layer]]?
- How are changes approved?
- Can employees easily discover the glossary?
## Meeting Scenario
**Situation:** Two reports show 400,000 and 520,000 subscribers.
**Possible response:**
> "Before comparing the numbers, let's confirm what each report means by Subscriber. If one includes free registered accounts and the other only paid accounts, both numbers may be correct but the labels are misleading."
## Key Takeaways
- A Business Glossary is the organization's governed dictionary.
- It documents agreed business definitions and ownership.
- [[Semantic Layer]] implements definitions; the glossary documents meaning.
- Many data disagreements are actually definition disagreements.
## Related Concepts
- [[Semantic Layer]]
- [[Data Governance]]
- [[Data Owner]]
- [[Data Steward]]
- [[Metadata]]
- [[Data Catalog]]
- [[KPI]]
- [[Metric]]
@@ -0,0 +1,106 @@
**MD file:** `Data Catalog.md`
**Estimated reading time:** 1012 minutes
# Data Catalog
## Definition
A **Data Catalog** is a searchable inventory of an organization's data assets and their [[Metadata]].
It helps people discover what data exists, what it means, where it comes from, who owns it, and whether it is trusted.
## Simple Mental Model
> **A Data Catalog is Google for your organization's data.**
## How It Works
A Data Catalog may inventory:
- Databases
- Tables
- Columns
- Files
- Dashboards
- Reports
- Metrics
- Models
- Pipelines
- APIs
An entry might show owner, status, description, grain, source, refresh, quality, and downstream use.
## Example
Search:
`Revenue`
Results:
```text
Net Advertising Revenue [Certified]
Subscription Revenue [Certified]
Gross Advertising Revenue
Revenue Forecast [Restricted]
```
Each result includes an owner and business definition.
## How It Fits Into the Bigger Picture
```text
[[Metadata]]
Collected and organized in
[[Data Catalog]]
Datasets + Metrics + Reports + Owners
```
The catalog may surface [[Data Lineage]], [[Data Quality]], [[Business Glossary]], [[Data Owner]], and [[Data Steward]] information.
## My Company / Real-World Context
A new employee should be able to search `Video Completion Rate` and see its definition, owner, dataset, source system, refresh, and dashboards that use it.
## CTO Perspective
Buying a catalog tool does not create governance. A catalog full of stale descriptions is only expensive documentation.
The hard part is the operating model: who maintains it, who certifies assets, what "trusted" means, and how deprecation works.
### Questions to Ask
- Can employees discover trusted data without knowing who created it?
- How do we mark certified vs experimental datasets?
- Who maintains catalog entries?
- Does the catalog show [[Data Lineage]]?
- Does it expose [[Data Quality]] status?
- Are people actually using it?
## Meeting Scenario
**Situation:** Someone proposes an expensive enterprise Data Catalog because "every mature organization needs one."
**Possible response:**
> "Before selecting a platform, let's define the problem. Are we struggling with discoverability, ownership, duplicated datasets, lineage, business definitions, or all of these? Then we can determine the lightest solution that meets the need."
## Key Takeaways
- A Data Catalog is a searchable inventory of data assets and metadata.
- It supports discoverability, self-service, and governance.
- A tool alone does not create governance.
- Operating model and adoption matter more than the interface.
## Related Concepts
- [[Metadata]]
- [[Data Governance]]
- [[Data Lineage]]
- [[Data Quality]]
- [[Business Glossary]]
- [[Data Owner]]
- [[Data Steward]]
@@ -0,0 +1,128 @@
**MD file:** `Data Contract.md`
**Estimated reading time:** 1113 minutes
# Data Contract
## Definition
A **Data Contract** is an explicit agreement between a data producer and data consumers about the structure, meaning, quality, and reliability of shared data.
## Simple Mental Model
> **A Data Contract is an API contract for data.**
A producer should not unexpectedly change data that downstream consumers rely on.
## How It Works
A contract might specify:
```text
Dataset: Viewing Events
Owner: Streaming Platform
Required fields:
event_id: string
content_id: string
event_time: timestamp
watch_time_ms: integer
platform: string
Freshness:
Available within 15 minutes
Quality:
event_id must be unique
content_id NULL rate < 0.5%
Breaking changes:
60-day notice required
```
## Example
Without a contract:
```text
watch_time_ms renamed
Pipeline breaks
Gold stops updating
Executive dashboard stale
```
With a contract:
```text
Proposed breaking change
Consumers notified
[[Impact Analysis]]
Migration planned
Consumers migrate
Old field removed
```
## How It Fits Into the Bigger Picture
Contracts can define [[Metadata]], expected [[Data Quality]], ownership, and change rules.
[[Data Lineage]] shows who depends on the contract.
## My Company / Real-World Context
If analytics depends on external fields such as `content_id`, `watch_time`, `device`, and `event_time`, important assumptions should be explicit in vendor or internal integration agreements.
## CTO Perspective
Data Contracts are about managing dependencies and accountability.
Ask:
> **What are downstream teams allowed to rely on, and how do producers communicate breaking changes?**
Use stronger contracts where data is critical, highly reused, cross-team, or vendor-controlled.
### Questions to Ask
- Who produces this data?
- Who consumes it?
- What fields and semantics are guaranteed?
- What quality and freshness are expected?
- What counts as a breaking change?
- How much notice is required?
- Can [[Data Lineage]] identify affected consumers?
- Who approves contract changes?
## Meeting Scenario
**Situation:** An internal application team wants to rename analytics fields during a release.
**Possible response:**
> "Before changing the fields, let's identify downstream dependencies and treat this as a data-interface change. We need to agree on the new contract, provide a migration period, and verify affected metrics and reports before removing the old fields."
## Key Takeaways
- A Data Contract defines expectations between producers and consumers.
- It can cover schema, meaning, freshness, quality, ownership, and change management.
- Contracts reduce unexpected downstream breakage.
- [[Data Lineage]] helps identify affected consumers.
- Treat important shared data interfaces with similar discipline to APIs.
## Related Concepts
- [[Data Lineage]]
- [[Data Quality]]
- [[Metadata]]
- [[Data Governance]]
- [[Impact Analysis]]
- [[Data Observability]]
- [[Schema Evolution]]
- [[API Contract]]
@@ -0,0 +1,105 @@
**MD file:** `Data Observability.md`
**Estimated reading time:** 1012 minutes
# Data Observability
## Definition
**Data Observability** is the ability to understand the health of data systems and quickly detect, diagnose, and resolve data problems.
## Simple Mental Model
> **Monitoring tells you that a pipeline ran. Data Observability tells you whether the data produced by that pipeline still makes sense.**
## How It Works
Common signals include:
- Freshness
- Volume
- Schema changes
- Distribution changes
- Quality-rule violations
- Downstream impact through lineage
Example:
```text
NULL platform_id
Normal: 0.2%
Today: 46%
```
## Example
A pipeline reports success but processes only 3.2M events when the normal range is 8M10M.
Observability should alert before executives consume the resulting dashboard.
## How It Fits Into the Bigger Picture
```text
[[Data Quality]]
Rules / expectations
+
[[Data Lineage]]
Dependencies
+
Operational telemetry
[[Data Observability]]
Detect + diagnose + alert
```
## My Company / Real-World Context
Useful signals could include:
- Viewing events received/hour
- Missing content ID percentage
- Unknown platform percentage
- Time since last successful load
- Source-to-Gold processing delay
## CTO Perspective
Ask:
> **How do we know our data is broken before a business user tells us?**
Avoid alerting on everything. Prioritize executive KPIs, revenue, compliance, and critical operational dashboards.
### Questions to Ask
- How quickly do we detect stale or incomplete data?
- Do we monitor volume and distribution, not just pipeline status?
- Which critical datasets have automated checks?
- Can alerts show downstream impact through [[Data Lineage]]?
- Who owns incident response?
## Meeting Scenario
**Situation:** Executives repeatedly discover broken dashboards before the data team does.
**Possible response:**
> "The recurring issue isn't only data quality; it's detection. We need observability on critical data products so freshness, volume, schema, and key quality conditions are checked automatically."
## Key Takeaways
- Data Observability detects and diagnoses unhealthy data systems.
- A successful pipeline does not guarantee healthy data.
- Freshness, volume, schema, distributions, and quality are common signals.
- [[Data Lineage]] helps identify downstream impact.
- Prioritize observability based on business criticality.
## Related Concepts
- [[Data Quality]]
- [[Data Lineage]]
- [[Data Governance]]
- [[Metadata]]
- [[Data Contract]]
- [[Monitoring]]
- [[Data Incident]]
+104
View File
@@ -0,0 +1,104 @@
**MD file:** `Data Owner.md`
**Estimated reading time:** 810 minutes
# Data Owner
## Definition
A **Data Owner** is the person or business role accountable for a data domain, dataset, or important business definition.
The Data Owner has authority over meaning, acceptable use, access, quality expectations, retention, and important business rules.
## Simple Mental Model
> **The Data Owner decides what the data means and what should happen with it.**
IT may host the system. Data Engineering may build the pipeline. An analyst may build the dashboard. None of them automatically owns the business meaning.
## How It Works
Example:
```text
Employee Data
Business owner: HR
IT: operates systems
Data team: pipelines and reporting
```
Ownership is accountability, not technical administration.
## Example
Marketing and Product disagree on `Active User`.
The data team should not silently choose the definition because it owns Power BI.
A Data Owner should approve the business definition, which can then flow into [[Business Glossary]] and [[Semantic Layer]].
## How It Fits Into the Bigger Picture
```text
[[Data Governance]]
[[Data Owner]]
Accountability / decision authority
[[Data Steward]]
Day-to-day quality and maintenance
```
## My Company / Real-World Context
Potential ownership could look like:
- Advertising Revenue → Finance / Advertising business leader
- Employee Data → HR
- Content Metadata → Programming / Content
- Streaming Audience KPI → agreed business owner
Your Data Governance team can facilitate and implement without owning every business concept.
## CTO Perspective
A common failure is confusing system ownership with data ownership.
A useful question is:
> **Who has the authority to make the final business decision about this data?**
If nobody can answer, ownership is unclear.
### Questions to Ask
- Who is accountable for this data?
- Who approves its business definition?
- Who approves access?
- Who establishes acceptable quality?
- Is the owner a business role or merely the person who built the system?
- Is ownership documented?
## Meeting Scenario
**Situation:** Marketing and Product ask your data team to decide the definition of Active User.
**Possible response:**
> "Data can quantify the options and explain the consequences, but this is ultimately a business-definition decision. We need an accountable owner who can approve the definition, after which we'll implement it consistently."
## Key Takeaways
- A Data Owner is accountable for business meaning and appropriate management.
- Ownership is not the same as building or hosting the system.
- Business departments should normally own business meaning.
- [[Data Governance]] should make ownership explicit.
## Related Concepts
- [[Data Steward]]
- [[Data Governance]]
- [[Business Glossary]]
- [[Metadata]]
- [[Data Catalog]]
- [[Semantic Layer]]
- [[KPI]]
@@ -0,0 +1,152 @@
**MD file:** `Data Quality.md`
**Estimated reading time:** 1215 minutes
# Data Quality
## Definition
**Data Quality** describes whether data is fit for the business purpose for which it is being used.
High-quality data is not simply data that exists or data that successfully loaded into a database. It must be sufficiently accurate, complete, consistent, timely, valid, and unique for the decisions being made from it.
[[Data Governance]] establishes the expectations and accountability for quality. Data Quality measures whether the data actually meets those expectations.
## Simple Mental Model
> **Data Governance defines what "good" means. Data Quality tells us whether today's data is actually good.**
A pipeline can run successfully and still produce poor-quality data.
```text
Pipeline status: SUCCESS
Rows loaded: 1,000,000
Half the platform values are NULL
Technically successful
Business quality: BAD
```
## How It Works
Common dimensions of Data Quality include:
### Accuracy
Does the data correctly represent reality?
### Completeness
Is required data present?
### Consistency
Is the same concept represented the same way across systems?
### Timeliness
Is the data available when the business needs it?
### Validity
Does the data follow expected rules?
### Uniqueness
Are duplicate records creating false counts?
## Example
Suppose Power BI shows:
> Yesterday's watch hours dropped by 35%.
Before concluding that audience behavior changed, the team checks quality:
```text
Source events received yesterday: 6.4M
Normal daily range: 9M10M
```
The dashboard calculation may be correct. The real problem is incomplete source data.
## How It Fits Into the Bigger Picture
```text
Source
[[ETL vs ELT]]
[[Medallion Architecture]]
Bronze
Silver ← quality checks often become critical here
Gold
[[Semantic Layer]]
Power BI
```
[[Data Lineage]] helps locate where a quality problem entered the chain.
## My Company / Real-World Context
For a media company, useful quality checks could include:
- Daily viewing-event volume
- Percentage of missing content IDs
- Duplicate event rate
- Percentage of unknown platforms
- Missing demographic percentage
- Data refresh completion time
- Difference between source-system totals and governed analytics totals
A high "Unknown" percentage is not automatically poor quality. The key question is whether it is expected due to consent, identity availability, or platform limitations, or caused by a collection problem.
## CTO Perspective
Important datasets can have explicit expectations such as:
```text
Daily refresh completed by 07:00
NULL content_id < 0.5%
Duplicate events < 0.1%
Source-vs-ingested row variance < 2%
```
The objective is not perfect data. The objective is quality appropriate to the business decision and risk.
### Questions to Ask
- What quality dimensions matter for this dataset?
- What thresholds define acceptable quality?
- Who owns those thresholds?
- Are quality checks automated?
- Who gets alerted when a check fails?
- Can we trace the issue through [[Data Lineage]]?
- What business decisions are affected when quality fails?
## Meeting Scenario
**Situation:** An executive says, "The dashboard must be wrong. Audience dropped 30% yesterday."
**Possible response:**
> "Before we interpret this as an audience change, let's validate the quality of the underlying data. I want to confirm source volume, pipeline completeness, and whether any upstream fields changed before we conclude that the business actually moved."
## Key Takeaways
- Data Quality means data is fit for its intended business purpose.
- Accuracy, completeness, consistency, timeliness, validity, and uniqueness are common quality dimensions.
- A successful pipeline can still produce poor-quality data.
- [[Data Governance]] defines expectations; Data Quality measures whether they are met.
- [[Data Lineage]] helps identify where problems originated.
## Related Concepts
- [[Data Governance]]
- [[Data Lineage]]
- [[Data Observability]]
- [[Metadata]]
- [[Data Contract]]
- [[Medallion Architecture]]
- [[Semantic Layer]]
- [[KPI]]
@@ -0,0 +1,91 @@
**MD file:** `Data Steward.md`
**Estimated reading time:** 810 minutes
# Data Steward
## Definition
A **Data Steward** is responsible for the day-to-day care, quality, definition, and proper use of data within a governed domain.
While the [[Data Owner]] is accountable and has decision authority, the steward helps make governance operational.
## Simple Mental Model
> **Owner = accountable. Steward = keeps it working properly day to day.**
## How It Works
A Data Steward may:
- Review quality issues
- Maintain definitions
- Validate classifications
- Coordinate corrections
- Update metadata
- Resolve routine questions
- Escalate policy decisions to the Data Owner
## Example
If the organization defines Roku, Apple TV, Android TV, and Fire TV as `Connected TV`, the owner approves that standard.
The steward helps detect inconsistent classifications and ensures future records follow the standard.
## How It Fits Into the Bigger Picture
```text
[[Data Governance]]
[[Data Owner]]
[[Data Steward]]
[[Data Quality]] + [[Metadata]] + standards
```
## My Company / Real-World Context
You may not need formal "Data Steward" job titles. Stewardship can be assigned to existing roles such as analysts or operations staff.
The key is that someone is clearly responsible for routine health and consistency.
## CTO Perspective
Governance structures can look great on PowerPoint while nobody actually performs the work.
Ask:
> **Who notices when this data becomes wrong, and what are they expected to do?**
### Questions to Ask
- Who performs day-to-day stewardship?
- What quality issues do they monitor?
- What can they decide themselves?
- What must be escalated to the [[Data Owner]]?
- Is stewardship part of their real workload?
- Who updates [[Metadata]] and [[Business Glossary]] entries?
## Meeting Scenario
**Situation:** Stewardship is assigned on paper, but data classifications remain inconsistent.
**Possible response:**
> "We have stewardship assigned, but the recurring issue suggests the operational responsibility isn't clear. Let's define what the steward monitors, what triggers action, and when the owner needs to be involved."
## Key Takeaways
- Data Stewards perform day-to-day governance.
- Owners are accountable; stewards operationalize standards.
- Stewardship does not require a dedicated job title.
- Governance should be lightweight enough to actually operate.
## Related Concepts
- [[Data Owner]]
- [[Data Governance]]
- [[Data Quality]]
- [[Metadata]]
- [[Business Glossary]]
- [[Data Catalog]]
+127
View File
@@ -0,0 +1,127 @@
**MD file:** `KPI.md`
**Estimated reading time:** 1113 minutes
# KPI
## Definition
A **KPI**, or **Key Performance Indicator**, is a metric selected because it indicates progress toward an important business objective.
Every KPI is a metric. Not every metric is a KPI.
## Simple Mental Model
> **Metric = something we can measure. KPI = a measurement important enough to tell us whether we are succeeding.**
## How It Works
A strong KPI connects:
```text
Business Objective
Desired Outcome
KPI
Target
Actions / Decisions
```
## Example
If the strategic objective is streaming growth, executive KPIs might include:
- Monthly Active Viewers
- Total Watch Hours
- Viewer Retention
- Streaming Revenue
Other metrics can still matter operationally without being KPIs.
## Leading vs Lagging Indicators
**Lagging indicators** measure outcomes after they happen, such as revenue or churn.
**Leading indicators** may provide earlier signals, such as weekly engagement or trial activation.
## How It Fits Into the Bigger Picture
```text
[[Fact Table]]
Raw measures
[[Semantic Layer]]
Governed metrics
[[Business Glossary]]
Agreed meaning
KPI
Power BI
Management decision
```
## My Company / Real-World Context
Potential KPIs could include:
- Monthly Active Viewers
- Total Watch Hours
- Viewer Retention
- Video Start Success Rate
- Advertising Revenue
- WCAG Compliance Rate
- SLA Compliance
If everything is "key," nothing is key.
## CTO Perspective
Bad executive dashboards display whatever is easy to measure.
Good dashboards display what is important to manage.
A CTO should challenge output metrics that do not demonstrate business value.
### Questions to Ask
- What business objective does this KPI represent?
- Why is it "key"?
- Who owns it?
- What is the target?
- What decision changes when this KPI changes?
- Is it leading or lagging?
- Can we trust the underlying [[Data Quality]]?
- Are we measuring output or outcome?
## Meeting Scenario
**Situation:** A department proposes 18 more metrics for the executive dashboard.
**Possible response:**
> "These may all be useful metrics, but which of them directly indicate whether we're achieving our strategic objectives, and what decisions would leadership make differently based on them?"
## Key Takeaways
- Every KPI is a metric, but not every metric is a KPI.
- KPIs should connect directly to business objectives.
- Leading indicators provide earlier signals; lagging indicators measure outcomes.
- Executive dashboards should focus attention.
- CTOs should distinguish output from outcome.
## Related Concepts
- [[Metric]]
- [[Measure]]
- [[Semantic Layer]]
- [[Business Glossary]]
- [[Data Governance]]
- [[Data Quality]]
- [[Data Lineage]]
- [[Leading Indicator]]
- [[Lagging Indicator]]
@@ -0,0 +1,103 @@
**MD file:** `Master Data Management.md`
**Estimated reading time:** 1113 minutes
# Master Data Management
## Definition
**Master Data Management (MDM)** is the discipline of creating and maintaining consistent, authoritative representations of important business entities across systems.
Common master-data domains include Customer, Product, Employee, Supplier, Location, and Content.
## Simple Mental Model
> **MDM answers: "What is the trusted identity of this thing across the company?"**
## How It Works
Example:
```text
CMS: Show ID = 784
JW Player: Media ID = ABC91
Advertising: Content ID = 55129
Analytics: Content Key = 9031
```
These may all refer to the same program.
MDM establishes mappings and an authoritative representation.
## Example
```text
CMS
"The Great Show - S1E1"
Video Platform
"great_show_ep01"
Programming
Program ID 81817
Advertising
Asset 500992
```
A governed master-content entity can map all of them to one canonical identity.
## How It Fits Into the Bigger Picture
MDM is related to [[Dimension Table]] and [[Data Governance]], but it is broader than analytics.
Master Data can serve operational and analytical systems.
## My Company / Real-World Context
Content is a likely MDM challenge in a media organization because the same content may have identifiers in CMS, video, OTT, linear scheduling, advertising, and analytics systems.
## CTO Perspective
MDM initiatives can become enormous and expensive.
Start with:
> **Which business entity is causing enough inconsistency that centralized management would create meaningful value?**
Do not start with purchasing an MDM platform.
### Questions to Ask
- Which entities exist in multiple systems?
- Which system is authoritative for each attribute?
- How are duplicates resolved?
- How are identifiers mapped?
- Who is the [[Data Owner]]?
- Do we need full MDM tooling or a simpler governed approach?
## Meeting Scenario
**Situation:** Your team spends significant time manually mapping content IDs across systems.
**Possible response:**
> "We clearly have a master-data problem around content identity. Let's define which system owns each attribute and the canonical identity we need before deciding what technology is required."
## Key Takeaways
- MDM manages authoritative identities across systems.
- It addresses inconsistency and duplication.
- Master Data is broader than analytical [[Dimension Table|Dimension Tables]].
- [[Data Governance]] determines ownership and authority.
- Solve the entity problem before choosing the platform.
## Related Concepts
- [[Data Governance]]
- [[Data Owner]]
- [[Dimension Table]]
- [[Natural Key]]
- [[Surrogate Key]]
- [[Metadata]]
- [[Data Quality]]
- [[Entity Resolution]]
+123
View File
@@ -0,0 +1,123 @@
**MD file:** `Metadata.md`
**Estimated reading time:** 911 minutes
# Metadata
## Definition
**Metadata** is data that describes other data.
It provides context needed to understand, discover, govern, and operate datasets.
## Simple Mental Model
> **Data is the book. Metadata is the title, author, table of contents, publication date, and description on the back.**
## How It Works
Metadata can be grouped into several categories.
### Technical Metadata
Table names, columns, data types, keys, schemas.
### Business Metadata
Business definitions, owners, descriptions, certified use.
### Operational Metadata
Refresh time, pipeline duration, row counts, success/failure.
### Governance Metadata
Owner, steward, classification, retention, access rules.
## Example
Without metadata:
```text
tbl_vw_agg_final_v2
dt
uid
cnt
sec
src
```
With metadata:
```text
Name: Daily Viewing Aggregate
Purpose: Daily content consumption reporting
Source: JW Player viewing events
Grain: One row per user/content/day
Owner: Audience team
Refresh: Daily 05:00
sec: Total valid watch seconds
```
## How It Fits Into the Bigger Picture
[[Data Lineage]] uses metadata to understand relationships.
[[Data Governance]] uses metadata to record ownership and classification.
[[Data Quality]] uses metadata to record rules and results.
[[Data Catalog]] makes metadata searchable.
## My Company / Real-World Context
Useful metadata could identify which audience dataset is certified, who owns it, how often it refreshes, and which source systems feed it.
## CTO Perspective
A mature organization moves knowledge from:
```text
People's heads
Documented metadata
Searchable organizational knowledge
```
The useful question is:
> **Is the metadata complete, current, discoverable, and actually used?**
### Questions to Ask
- Who owns this dataset?
- What business purpose does it serve?
- What is its [[Grain]]?
- Where does it come from?
- How often is it refreshed?
- Is it certified for business use?
- Can users discover this information without asking the original developer?
## Meeting Scenario
**Situation:** A new analyst finds four datasets that appear to contain audience information.
**Possible response:**
> "If the only way to know which dataset is trusted is tribal knowledge, that's a scalability problem. Let's make the owner, purpose, status, grain, source, and refresh information discoverable as metadata."
## Key Takeaways
- Metadata is data about data.
- Technical metadata explains structure.
- Business metadata explains meaning.
- Operational metadata explains processing and freshness.
- Governance metadata explains ownership and controls.
- [[Data Catalog]] makes metadata discoverable.
## Related Concepts
- [[Data Catalog]]
- [[Data Governance]]
- [[Data Lineage]]
- [[Data Quality]]
- [[Business Glossary]]
- [[Data Owner]]
- [[Data Steward]]
- [[Grain]]
@@ -0,0 +1,197 @@
# CTO Academy - Data
This is the canonical reading order for the Data module.
## Progress
- [ ] Lesson 01 - [[Medallion Architecture]]
- [ ] Lesson 02 - [[Data Lake vs Data Warehouse vs Lakehouse]]
- [ ] Lesson 03 - [[ETL vs ELT]]
- [ ] Lesson 04 - [[Fact Table]]
- [ ] Lesson 05 - [[Dimension Table]]
- [ ] Lesson 06 - [[Star Schema]]
- [ ] Lesson 07 - [[Snowflake Schema]]
- [ ] Lesson 08 - [[Semantic Layer]]
- [ ] Lesson 09 - [[Data Lineage]]
- [ ] Lesson 10 - [[Data Governance]]
- [ ] Lesson 11 - [[Data Quality]]
- [ ] Lesson 12 - [[Metadata]]
- [ ] Lesson 13 - [[Data Catalog]]
- [ ] Lesson 14 - [[Data Owner]]
- [ ] Lesson 15 - [[Data Steward]]
- [ ] Lesson 16 - [[Business Glossary]]
- [ ] Lesson 17 - [[KPI]]
- [ ] Lesson 18 - [[Master Data Management]]
- [ ] Lesson 19 - [[Data Observability]]
- [ ] Lesson 20 - [[Data Contract]]
---
## Phase 1 - Data Architecture Foundations
### Lesson 01 - [[Medallion Architecture]]
Understand the Bronze → Silver → Gold model and how data matures through the platform.
### Lesson 02 - [[Data Lake vs Data Warehouse vs Lakehouse]]
Understand the major analytical storage architectures and when each is appropriate.
### Lesson 03 - [[ETL vs ELT]]
Understand where transformation happens in the data pipeline and why modern architectures increasingly use ELT.
---
## Phase 2 - Analytical Data Modeling
### Lesson 04 - [[Fact Table]]
Understand how business events and measurements are represented.
### Lesson 05 - [[Dimension Table]]
Understand how descriptive context is attached to business events.
### Lesson 06 - [[Star Schema]]
Understand the standard dimensional model used by analytical and BI systems.
### Lesson 07 - [[Snowflake Schema]]
Understand normalization of dimensions and the trade-off between simplicity and reduced duplication.
---
## Phase 3 - Business Meaning and Trust
### Lesson 08 - [[Semantic Layer]]
Understand how organizations create shared and reusable business definitions.
### Lesson 09 - [[Data Lineage]]
Understand where data comes from, how it changes, where it goes, and how to perform impact analysis.
### Lesson 10 - [[Data Governance]]
Understand the people, policies, ownership, standards, and processes that make organizational data trustworthy.
---
## Phase 4 - Operationalizing Data Governance
### Lesson 11 - [[Data Quality]]
Understand how accuracy, completeness, consistency, timeliness, validity, and uniqueness are measured.
### Lesson 12 - [[Metadata]]
Understand the information that describes datasets, fields, ownership, refresh schedules, and business meaning.
### Lesson 13 - [[Data Catalog]]
Understand how organizations make datasets, metrics, lineage, ownership, and metadata discoverable.
### Lesson 14 - [[Data Owner]]
Understand who is accountable for the business meaning and appropriate management of data.
### Lesson 15 - [[Data Steward]]
Understand who handles the day-to-day operational responsibilities of governed data.
### Lesson 16 - [[Business Glossary]]
Understand how an organization establishes shared definitions for important business concepts.
---
## Phase 5 - Managing Data as a Business Asset
### Lesson 17 - [[KPI]]
Understand the difference between measures, metrics, KPIs, outputs, outcomes, and leading/lagging indicators.
### Lesson 18 - [[Master Data Management]]
Understand how organizations establish authoritative identities for important entities across systems.
### Lesson 19 - [[Data Observability]]
Understand how data teams detect broken, stale, incomplete, or abnormal data before business users discover it.
### Lesson 20 - [[Data Contract]]
Understand how producers and consumers establish explicit expectations for shared data.
---
# Quizzes
- [[Quiz 01 - Data Foundations]]
---
# Mental Model
The technical path:
Source Systems
[[ETL vs ELT]]
[[Medallion Architecture]]
Bronze → Silver → Gold
[[Fact Table]] + [[Dimension Table]]
[[Star Schema]] / [[Snowflake Schema]]
[[Semantic Layer]]
Power BI
Business Decisions
The governance surrounding it:
[[Data Governance]]
├── [[Data Owner]]
├── [[Data Steward]]
├── [[Business Glossary]]
├── [[Metadata]]
├── [[Data Catalog]]
├── [[Data Quality]]
├── [[Data Lineage]]
└── [[Data Contract]]
The operational trust layer:
[[Data Quality]]
+
[[Data Lineage]]
+
[[Data Observability]]
Can we trust the data right now?
---
# CTO Questions to Remember
When presented with data, ask:
1. What business question are we trying to answer?
2. What does one row represent? → [[Fact Table]]
3. What is the [[Grain]]?
4. What dimensions provide context? → [[Dimension Table]]
5. What is the business definition? → [[Business Glossary]]
6. Who owns that definition? → [[Data Owner]]
7. Is the definition implemented consistently? → [[Semantic Layer]]
8. Where did the data come from? → [[Data Lineage]]
9. Can we trust it? → [[Data Quality]]
10. How would we know if it broke? → [[Data Observability]]
11. What happens downstream if the source changes? → [[Data Lineage]]
12. What are producers expected to guarantee? → [[Data Contract]]