vault backup: 2026-07-23 07:56:59

2026-07-23 07:56:59 -04:00
parent 1033a565cb
commit af0951d4db
11 changed files with 1355 additions and 0 deletions
@@ -0,0 +1,123 @@
+**MD file:** `Metadata.md`  
+**Estimated reading time:** 9–11 minutes
+
+# Metadata
+
+## Definition
+
+**Metadata** is data that describes other data.
+
+It provides context needed to understand, discover, govern, and operate datasets.
+
+## Simple Mental Model
+
+> **Data is the book. Metadata is the title, author, table of contents, publication date, and description on the back.**
+
+## How It Works
+
+Metadata can be grouped into several categories.
+
+### Technical Metadata
+Table names, columns, data types, keys, schemas.
+
+### Business Metadata
+Business definitions, owners, descriptions, certified use.
+
+### Operational Metadata
+Refresh time, pipeline duration, row counts, success/failure.
+
+### Governance Metadata
+Owner, steward, classification, retention, access rules.
+
+## Example
+
+Without metadata:
+
+```text
+tbl_vw_agg_final_v2
+dt
+uid
+cnt
+sec
+src
+```
+
+With metadata:
+
+```text
+Name: Daily Viewing Aggregate
+Purpose: Daily content consumption reporting
+Source: JW Player viewing events
+Grain: One row per user/content/day
+Owner: Audience team
+Refresh: Daily 05:00
+sec: Total valid watch seconds
+```
+
+## How It Fits Into the Bigger Picture
+
+[[Data Lineage]] uses metadata to understand relationships.
+
+[[Data Governance]] uses metadata to record ownership and classification.
+
+[[Data Quality]] uses metadata to record rules and results.
+
+[[Data Catalog]] makes metadata searchable.
+
+## My Company / Real-World Context
+
+Useful metadata could identify which audience dataset is certified, who owns it, how often it refreshes, and which source systems feed it.
+
+## CTO Perspective
+
+A mature organization moves knowledge from:
+
+```text
+People's heads
+      ↓
+Documented metadata
+      ↓
+Searchable organizational knowledge
+```
+
+The useful question is:
+
+> **Is the metadata complete, current, discoverable, and actually used?**
+
+### Questions to Ask
+
+- Who owns this dataset?
+- What business purpose does it serve?
+- What is its [[Grain]]?
+- Where does it come from?
+- How often is it refreshed?
+- Is it certified for business use?
+- Can users discover this information without asking the original developer?
+
+## Meeting Scenario
+
+**Situation:** A new analyst finds four datasets that appear to contain audience information.
+
+**Possible response:**
+
+> "If the only way to know which dataset is trusted is tribal knowledge, that's a scalability problem. Let's make the owner, purpose, status, grain, source, and refresh information discoverable as metadata."
+
+## Key Takeaways
+
+- Metadata is data about data.
+- Technical metadata explains structure.
+- Business metadata explains meaning.
+- Operational metadata explains processing and freshness.
+- Governance metadata explains ownership and controls.
+- [[Data Catalog]] makes metadata discoverable.
+
+## Related Concepts
+
+- [[Data Catalog]]
+- [[Data Governance]]
+- [[Data Lineage]]
+- [[Data Quality]]
+- [[Business Glossary]]
+- [[Data Owner]]
+- [[Data Steward]]
+- [[Grain]]