**Estimated reading time:** 10–12 minutes

## Definition

**Data Observability** is the ability to understand the health of data systems and quickly detect, diagnose, and resolve data problems.

## Simple Mental Model

> **Monitoring tells you that a pipeline ran. Data Observability tells you whether the data produced by that pipeline still makes sense.**

## How It Works

Common signals include:

- Freshness
- Volume
- Schema changes
- Distribution changes
- Quality-rule violations
- Downstream impact through lineage

Example:

```text
NULL platform_id
Normal: 0.2%
Today: 46%
```

## Example

A pipeline reports success but processes only 3.2M events when the normal range is 8M–10M.

Observability should alert before executives consume the resulting dashboard.

## How It Fits Into the Bigger Picture

```text
[[Data Quality]]
Rules / expectations
       +
[[Data Lineage]]
Dependencies
       +
Operational telemetry
       ↓
[[Data Observability]]
       ↓
Detect + diagnose + alert
```

## My Company / Real-World Context

Useful signals could include:

- Viewing events received/hour
- Missing content ID percentage
- Unknown platform percentage
- Time since last successful load
- Source-to-Gold processing delay

## CTO Perspective

Ask:

> **How do we know our data is broken before a business user tells us?**

Avoid alerting on everything. Prioritize executive KPIs, revenue, compliance, and critical operational dashboards.

### Questions to Ask

- How quickly do we detect stale or incomplete data?
- Do we monitor volume and distribution, not just pipeline status?
- Which critical datasets have automated checks?
- Can alerts show downstream impact through [[Data Lineage]]?
- Who owns incident response?

## Meeting Scenario

**Situation:** Executives repeatedly discover broken dashboards before the data team does.

**Possible response:**

> "The recurring issue isn't only data quality; it's detection. We need observability on critical data products so freshness, volume, schema, and key quality conditions are checked automatically."

## Key Takeaways

- Data Observability detects and diagnoses unhealthy data systems.
- A successful pipeline does not guarantee healthy data.
- Freshness, volume, schema, distributions, and quality are common signals.
- [[Data Lineage]] helps identify downstream impact.
- Prioritize observability based on business criticality.

## Related Concepts

- [[Data Quality]]
- [[Data Lineage]]
- [[Data Governance]]
- [[Metadata]]
- [[Data Contract]]
- [[Monitoring]]
- [[Data Incident]]