121 lines
2.8 KiB
Markdown
121 lines
2.8 KiB
Markdown
**Estimated reading time:** 9–11 minutes
|
||
|
||
## Definition
|
||
|
||
**Metadata** is data that describes other data.
|
||
|
||
It provides context needed to understand, discover, govern, and operate datasets.
|
||
|
||
## Simple Mental Model
|
||
|
||
> **Data is the book. Metadata is the title, author, table of contents, publication date, and description on the back.**
|
||
|
||
## How It Works
|
||
|
||
Metadata can be grouped into several categories.
|
||
|
||
### Technical Metadata
|
||
Table names, columns, data types, keys, schemas.
|
||
|
||
### Business Metadata
|
||
Business definitions, owners, descriptions, certified use.
|
||
|
||
### Operational Metadata
|
||
Refresh time, pipeline duration, row counts, success/failure.
|
||
|
||
### Governance Metadata
|
||
Owner, steward, classification, retention, access rules.
|
||
|
||
## Example
|
||
|
||
Without metadata:
|
||
|
||
```text
|
||
tbl_vw_agg_final_v2
|
||
dt
|
||
uid
|
||
cnt
|
||
sec
|
||
src
|
||
```
|
||
|
||
With metadata:
|
||
|
||
```text
|
||
Name: Daily Viewing Aggregate
|
||
Purpose: Daily content consumption reporting
|
||
Source: JW Player viewing events
|
||
Grain: One row per user/content/day
|
||
Owner: Audience team
|
||
Refresh: Daily 05:00
|
||
sec: Total valid watch seconds
|
||
```
|
||
|
||
## How It Fits Into the Bigger Picture
|
||
|
||
[[Data Lineage]] uses metadata to understand relationships.
|
||
|
||
[[Data Governance]] uses metadata to record ownership and classification.
|
||
|
||
[[Data Quality]] uses metadata to record rules and results.
|
||
|
||
[[Data Catalog]] makes metadata searchable.
|
||
|
||
## My Company / Real-World Context
|
||
|
||
Useful metadata could identify which audience dataset is certified, who owns it, how often it refreshes, and which source systems feed it.
|
||
|
||
## CTO Perspective
|
||
|
||
A mature organization moves knowledge from:
|
||
|
||
```text
|
||
People's heads
|
||
↓
|
||
Documented metadata
|
||
↓
|
||
Searchable organizational knowledge
|
||
```
|
||
|
||
The useful question is:
|
||
|
||
> **Is the metadata complete, current, discoverable, and actually used?**
|
||
|
||
### Questions to Ask
|
||
|
||
- Who owns this dataset?
|
||
- What business purpose does it serve?
|
||
- What is its [[Grain]]?
|
||
- Where does it come from?
|
||
- How often is it refreshed?
|
||
- Is it certified for business use?
|
||
- Can users discover this information without asking the original developer?
|
||
|
||
## Meeting Scenario
|
||
|
||
**Situation:** A new analyst finds four datasets that appear to contain audience information.
|
||
|
||
**Possible response:**
|
||
|
||
> "If the only way to know which dataset is trusted is tribal knowledge, that's a scalability problem. Let's make the owner, purpose, status, grain, source, and refresh information discoverable as metadata."
|
||
|
||
## Key Takeaways
|
||
|
||
- Metadata is data about data.
|
||
- Technical metadata explains structure.
|
||
- Business metadata explains meaning.
|
||
- Operational metadata explains processing and freshness.
|
||
- Governance metadata explains ownership and controls.
|
||
- [[Data Catalog]] makes metadata discoverable.
|
||
|
||
## Related Concepts
|
||
|
||
- [[Data Catalog]]
|
||
- [[Data Governance]]
|
||
- [[Data Lineage]]
|
||
- [[Data Quality]]
|
||
- [[Business Glossary]]
|
||
- [[Data Owner]]
|
||
- [[Data Steward]]
|
||
- [[Grain]]
|