129 lines
3.0 KiB
Markdown
129 lines
3.0 KiB
Markdown
**MD file:** `Data Contract.md`
|
||
**Estimated reading time:** 11–13 minutes
|
||
|
||
# Data Contract
|
||
|
||
## Definition
|
||
|
||
A **Data Contract** is an explicit agreement between a data producer and data consumers about the structure, meaning, quality, and reliability of shared data.
|
||
|
||
## Simple Mental Model
|
||
|
||
> **A Data Contract is an API contract for data.**
|
||
|
||
A producer should not unexpectedly change data that downstream consumers rely on.
|
||
|
||
## How It Works
|
||
|
||
A contract might specify:
|
||
|
||
```text
|
||
Dataset: Viewing Events
|
||
Owner: Streaming Platform
|
||
|
||
Required fields:
|
||
event_id: string
|
||
content_id: string
|
||
event_time: timestamp
|
||
watch_time_ms: integer
|
||
platform: string
|
||
|
||
Freshness:
|
||
Available within 15 minutes
|
||
|
||
Quality:
|
||
event_id must be unique
|
||
content_id NULL rate < 0.5%
|
||
|
||
Breaking changes:
|
||
60-day notice required
|
||
```
|
||
|
||
## Example
|
||
|
||
Without a contract:
|
||
|
||
```text
|
||
watch_time_ms renamed
|
||
↓
|
||
Pipeline breaks
|
||
↓
|
||
Gold stops updating
|
||
↓
|
||
Executive dashboard stale
|
||
```
|
||
|
||
With a contract:
|
||
|
||
```text
|
||
Proposed breaking change
|
||
↓
|
||
Consumers notified
|
||
↓
|
||
[[Impact Analysis]]
|
||
↓
|
||
Migration planned
|
||
↓
|
||
Consumers migrate
|
||
↓
|
||
Old field removed
|
||
```
|
||
|
||
## How It Fits Into the Bigger Picture
|
||
|
||
Contracts can define [[Metadata]], expected [[Data Quality]], ownership, and change rules.
|
||
|
||
[[Data Lineage]] shows who depends on the contract.
|
||
|
||
## My Company / Real-World Context
|
||
|
||
If analytics depends on external fields such as `content_id`, `watch_time`, `device`, and `event_time`, important assumptions should be explicit in vendor or internal integration agreements.
|
||
|
||
## CTO Perspective
|
||
|
||
Data Contracts are about managing dependencies and accountability.
|
||
|
||
Ask:
|
||
|
||
> **What are downstream teams allowed to rely on, and how do producers communicate breaking changes?**
|
||
|
||
Use stronger contracts where data is critical, highly reused, cross-team, or vendor-controlled.
|
||
|
||
### Questions to Ask
|
||
|
||
- Who produces this data?
|
||
- Who consumes it?
|
||
- What fields and semantics are guaranteed?
|
||
- What quality and freshness are expected?
|
||
- What counts as a breaking change?
|
||
- How much notice is required?
|
||
- Can [[Data Lineage]] identify affected consumers?
|
||
- Who approves contract changes?
|
||
|
||
## Meeting Scenario
|
||
|
||
**Situation:** An internal application team wants to rename analytics fields during a release.
|
||
|
||
**Possible response:**
|
||
|
||
> "Before changing the fields, let's identify downstream dependencies and treat this as a data-interface change. We need to agree on the new contract, provide a migration period, and verify affected metrics and reports before removing the old fields."
|
||
|
||
## Key Takeaways
|
||
|
||
- A Data Contract defines expectations between producers and consumers.
|
||
- It can cover schema, meaning, freshness, quality, ownership, and change management.
|
||
- Contracts reduce unexpected downstream breakage.
|
||
- [[Data Lineage]] helps identify affected consumers.
|
||
- Treat important shared data interfaces with similar discipline to APIs.
|
||
|
||
## Related Concepts
|
||
|
||
- [[Data Lineage]]
|
||
- [[Data Quality]]
|
||
- [[Metadata]]
|
||
- [[Data Governance]]
|
||
- [[Impact Analysis]]
|
||
- [[Data Observability]]
|
||
- [[Schema Evolution]]
|
||
- [[API Contract]]
|