From 2f654ebaec749c31b82168bcd0f3824c57c338ce Mon Sep 17 00:00:00 2001 From: Amadou Date: Tue, 21 Jul 2026 11:36:51 -0400 Subject: [PATCH] vault backup: 2026-07-21 11:36:51 --- .DS_Store | Bin 0 -> 6148 bytes .gitignore | 2 + .gitignore.json | 1 + .obsidian/plugins/iconic/data.json.backup2 | 24 +- .obsidian/workspace.json | 38 +- 10 Knowledge/CTO Academy/Data/Data Lineage.md | 673 ++++++++++++++ .../CTO Academy/Data/Quizzes/.gitkeep | 0 .../Quizzes/Quiz 01 - Data Foundations.md | 878 ++++++++++++++++++ 8 files changed, 1612 insertions(+), 4 deletions(-) create mode 100644 .DS_Store create mode 100644 .gitignore create mode 100644 .gitignore.json create mode 100644 10 Knowledge/CTO Academy/Data/Data Lineage.md create mode 100644 10 Knowledge/CTO Academy/Data/Quizzes/.gitkeep create mode 100644 10 Knowledge/CTO Academy/Data/Quizzes/Quiz 01 - Data Foundations.md diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..2bffbbe4b25208ca092f3f534fcb17af1937c978 GIT binary patch literal 6148 zcmeHKU5gVj6usH4?$kx-gTg*Zz}K!*Z4tzmVXSY9f}`t$N}bsm*THF0W-^wg6y{m~ zkiW#=P@6v>6;B)Ruwa!*Op5D|;p{2|eRhyoOiy<4cRFzwf6%l6zz56H|M zCFN97N+MfsMf(A-fLCDG6wrIOPY*~^Mh*7v@BP~>vPx#~{>Tgd!obrXrPTEG* znCFBNx&T9gH>Z$J!K|o7Th9~H6sMv>3F!^`1=VIWg+GPn5|P4be*v$KmspME3o!K@ zFE@W9;`mC6m@1q21C@CK0QKJpV`%HQgywLEk;Bd+JTPXdKucxriNP!#Cl*=swO7C^u&Kbl?GE()|NQUo|4owbc?G-z|CIu)H%+E7 zQgV0eR&x5T Where did this number come from? + +> Which source system produced it? + +> What transformations were applied? + +> Which reports depend on it? + +A simple lineage path might look like: + +```text +JW Player + ↓ +Raw Viewing Events + ↓ +Bronze + ↓ +Silver + ↓ +Gold + ↓ +[[Fact Table]] + ↓ +[[Semantic Layer]] + ↓ +Power BI Report +``` + +Data Lineage documents that path. + +--- + +## Simple Mental Model + +Think: + +> **Data Lineage = The family tree and travel history of data.** + +It tells you: + +```text +Where it started +↓ +What happened to it +↓ +Where it ended up +``` + +For a KPI, lineage lets you trace backward: + +```text +Executive Dashboard + ↓ +Total Watch Hours + ↓ +Semantic Layer + ↓ +fact_viewing.watch_seconds + ↓ +Gold + ↓ +Silver + ↓ +Bronze + ↓ +JW Player +``` + +--- + +## How It Works + +Imagine a Power BI dashboard displays: + +> Total Watch Hours: 2.4 million + +Someone questions the number. + +Without Data Lineage, the investigation might look like: + +```text +Ask Power BI analyst +↓ +Check report logic +↓ +Ask data engineer +↓ +Check SQL +↓ +Ask platform team +↓ +Check source data +``` + +Nobody has a clear view of the full chain. + +With Data Lineage, you can trace: + +```text +Power BI +↓ +Metric: Total Watch Hours +↓ +[[Semantic Layer]] +↓ +SUM(watch_seconds) / 3600 +↓ +fact_viewing +↓ +Gold transformation +↓ +Silver viewing events +↓ +Bronze raw events +↓ +JW Player API +``` + +Now the investigation is much more structured. + +--- + +## Example + +Suppose your company changes video providers. + +You move from: + +```text +JW Player +``` + +to: + +```text +New Video Platform +``` + +You want to know: + +> Which dashboards will be affected? + +Data Lineage should help answer: + +```text +JW Player + ↓ +Viewing Pipeline + ↓ +fact_viewing + ↓ +Semantic Layer + ↓ +Total Watch Hours + ↓ +Executive Dashboard + ↓ +Content Performance Dashboard + ↓ +Marketing Dashboard +``` + +This is sometimes called **impact analysis**. + +Instead of discovering broken reports after migration, you can identify dependencies beforehand. + +--- + +## How It Fits Into the Bigger Picture + +Now the architecture you've been learning connects like this: + +```text +Source Systems + ↓ +[[ETL vs ELT]] + ↓ +[[Medallion Architecture]] + ↓ +Bronze + ↓ +Silver + ↓ +Gold + ↓ +[[Fact Table]] ++ +[[Dimension Table]] + ↓ +[[Star Schema]] +or +[[Snowflake Schema]] + ↓ +[[Semantic Layer]] + ↓ +Power BI +``` + +[[Data Lineage]] cuts **across the entire architecture**. + +It does not represent one layer. + +It describes the connections between all the layers. + +Conceptually: + +```text +SOURCE → TRANSFORMATION → MODEL → METRIC → REPORT +``` + +That entire chain is lineage. + +--- + +## Technical Lineage vs Business Lineage + +Data Lineage can exist at different levels. + +### Technical Lineage + +Focuses on systems, tables, columns, and transformations. + +Example: + +```text +jw_events.watch_time_ms +↓ +silver_viewing.watch_seconds +↓ +fact_viewing.watch_seconds +↓ +semantic_model.total_watch_hours +``` + +This is useful for: + +- Data engineers + +- Developers + +- Analysts + +- Troubleshooting + + +### Business Lineage + +Focuses on business meaning. + +Example: + +```text +JW Player Viewing Events +↓ +Validated Viewing Data +↓ +Viewing Fact +↓ +Total Watch Hours +↓ +Executive Audience Dashboard +``` + +This is easier for: + +- Managers + +- Executives + +- Data owners + +- Business stakeholders + + +A mature data environment often needs both. + +--- + +## Column-Level Lineage + +Lineage can become very detailed. + +Imagine the final metric: + +```text +Total Watch Hours +``` + +comes from: + +```text +watch_time_ms +``` + +in the source system. + +The transformation could be: + +```text +JW Player +watch_time_ms + ↓ +divide by 1000 + ↓ +watch_seconds + ↓ +divide by 3600 + ↓ +Total Watch Hours +``` + +Column-level lineage tracks that exact transformation. + +This is valuable when someone asks: + +> Why is the number different from the source platform? + +You can see whether: + +- Units were converted + +- Filters were applied + +- Records were excluded + +- Duplicates were removed + +- Business rules were introduced + + +--- + +## Data Lineage and Data Quality + +Imagine a dashboard suddenly shows: + +```text +Total Watch Hours +↓ 40% +``` + +The first question might be: + +> Did audience actually drop? + +But the problem could be: + +```text +JW Player +↓ +API changed +↓ +Pipeline failed +↓ +Silver missing records +↓ +Gold incomplete +↓ +Dashboard shows lower number +``` + +Without lineage, a technical issue can look like a business event. + +With lineage, the team can trace the problem upstream. + +This connects directly to [[Data Quality]]. + +A number can be mathematically correct based on the data it received, while the underlying data is incomplete. + +--- + +## Data Lineage and Data Governance + +[[Data Governance]] asks questions such as: + +> Who owns this data? + +> What does this KPI mean? + +> Who can access it? + +> Can we trust it? + +Data Lineage helps answer: + +> Where did it come from? + +Together: + +```text +[[Data Governance]] + + +[[Data Lineage]] + + +[[Semantic Layer]] + = +More trustworthy analytics +``` + +Governance without lineage can define what a metric should mean without proving how it was produced. + +Lineage without governance can show the technical path without explaining who owns the definition. + +You usually need both. + +--- + +## My Company / Real-World Context + +Imagine the CEO asks: + +> Why does the Power BI dashboard show 1.8 million watch hours while JW Player shows 2.1 million? + +A weak process might involve several days of investigation. + +You ask: + +- Power BI analyst + +- Data engineer + +- JW Player team + +- Vendor + +- Data Governance + + +Each person knows one piece. + +With proper lineage, you could trace: + +```text +JW Player Raw Metric +2.1M hours + ↓ +Bronze + ↓ +Remove invalid events + ↓ +Silver + ↓ +Exclude internal traffic + ↓ +Gold + ↓ +Apply governed viewing definition + ↓ +Semantic Layer + ↓ +Power BI +1.8M hours +``` + +Now the difference may actually be intentional. + +The source platform might count all activity. + +Your governed KPI might exclude: + +- Internal testing + +- Bots + +- Invalid sessions + +- Duplicate events + + +The numbers are different because they answer different questions. + +Lineage helps explain why. + +--- + +## Another Real-World Example: Vendor Migration + +Imagine your company wants to replace JW Player. + +Before signing the new vendor, you should understand what depends on the current one. + +Lineage could reveal: + +```text +JW Player +├── Viewing Analytics +│ ├── Executive Dashboard +│ ├── Content Dashboard +│ └── Audience Report +│ +├── Advertising Metrics +│ └── Revenue Dashboard +│ +└── Content Metadata + ├── Website + └── Mobile App +``` + +Now the migration is no longer: + +> Replace JW Player. + +It is: + +> Replace JW Player while preserving all downstream dependencies. + +That is a much more accurate project scope. + +--- + +## CTO Perspective + +Data Lineage is important for a CTO because modern organizations have interconnected systems. + +A small source change can create unexpected downstream effects. + +The CTO-level questions are: + +> **"If this source changes, what breaks?"** + +> **"If this KPI is wrong, can we trace it back to the source?"** + +> **"Do we know which reports depend on this dataset?"** + +> **"Can we explain how an executive number was produced?"** + +Lineage is also important for: + +- Auditing + +- Compliance + +- Security + +- Data migrations + +- Vendor changes + +- Troubleshooting + +- Impact analysis + + +A mature organization should not depend entirely on one employee's memory to understand how critical data moves. + +The architecture itself should be documented or discoverable. + +--- + +### Questions to Ask + +- What is the original source of this data? + +- Which pipelines transform it? + +- Where does it move through Bronze, Silver, and Gold? + +- Which [[Fact Table|Fact Tables]] and [[Dimension Table|Dimension Tables]] depend on it? + +- Which metrics in the [[Semantic Layer]] use it? + +- Which reports consume those metrics? + +- What happens downstream if this source changes? + +- Can we trace a KPI back to the source field? + +- Are transformations documented? + +- Who owns each stage of the data flow? + +- Is lineage captured automatically or manually? + +- How current is our lineage documentation? + + +--- + +## Meeting Scenario + +**Situation:** + +A vendor announces that an API field your company uses will be removed in 60 days. + +The discussion immediately becomes: + +> "The data team needs to update the API integration." + +**Possible response:** + +> "Before we treat this only as an integration change, can we trace the lineage of that field? I want to know which transformations, metrics, and reports depend on it so we understand the full impact." + +The team discovers: + +```text +API Field +↓ +Silver Transformation +↓ +fact_viewing +↓ +Completion Rate +↓ +Three Power BI Dashboards +↓ +Monthly Executive Report +``` + +You could then respond: + +> "Good. Let's treat this as a dependency change across the full lineage, not just an API ticket. We need to update the source mapping, validate the transformation, confirm the KPI calculation, and test the affected reports before the vendor removes the field." + +You have turned a narrow technical task into a properly scoped change-management problem. + +--- + +## Key Takeaways + +- **Data Lineage** shows where data originates, how it changes, and where it is consumed. + +- Lineage crosses the entire data architecture rather than belonging to one layer. + +- Technical lineage focuses on tables, columns, and transformations. + +- Business lineage focuses on business concepts, metrics, and reports. + +- Column-level lineage can explain exactly how a KPI was calculated. + +- Lineage supports troubleshooting and [[Data Quality]] investigations. + +- Lineage enables impact analysis before source systems or vendors change. + +- [[Data Governance]] defines ownership and meaning; Data Lineage explains the path. + +- A CTO should be able to ask: **"If this changes, what downstream systems and decisions are affected?"** + + +## Related Concepts + +- [[Data Governance]] + +- [[Data Quality]] + +- [[Semantic Layer]] + +- [[Medallion Architecture]] + +- [[ETL vs ELT]] + +- [[Fact Table]] + +- [[Dimension Table]] + +- [[Star Schema]] + +- [[Data Pipeline]] + +- [[Metadata]] + +- [[Data Catalog]] + +- [[Impact Analysis]] + + +The natural next lesson is **Lesson 10: `Data Governance.md`**. Given that this is already part of your actual role, it will be interesting because we'll compare what **Data Governance formally means** against what you're currently doing in practice, including ownership, stewardship, KPI definitions, quality, lineage, and accountability. \ No newline at end of file diff --git a/10 Knowledge/CTO Academy/Data/Quizzes/.gitkeep b/10 Knowledge/CTO Academy/Data/Quizzes/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/10 Knowledge/CTO Academy/Data/Quizzes/Quiz 01 - Data Foundations.md b/10 Knowledge/CTO Academy/Data/Quizzes/Quiz 01 - Data Foundations.md new file mode 100644 index 0000000..cc84c55 --- /dev/null +++ b/10 Knowledge/CTO Academy/Data/Quizzes/Quiz 01 - Data Foundations.md @@ -0,0 +1,878 @@ + + +**Score: 20.5 / 24 — 85%** + +## Overall Assessment + +### Strongest Areas + +- [[Medallion Architecture]] + +- [[ETL vs ELT]] + +- [[Grain]] + +- [[Fact Table]] + +- [[Semantic Layer]] + +- CTO-level decision-making and architectural judgment + + +### Areas to Reinforce + +- [[Data Lake]] vs [[Data Warehouse]] + +- [[Fact Table]] vs [[Dimension Table]] + +- [[Star Schema]] vs [[Snowflake Schema]] + +- [[Data Lineage]] + + +Most of my mistakes were about precise definitions and architectural trade-offs rather than the overall architecture. + +--- + +# Section A - Multiple Choice + +## Question 1 + +**Question:** What is the best description of the Gold layer? + +A. The place where all raw source data is stored unchanged +B. The layer where data is cleaned technically but has no business meaning +C. The layer containing business-ready data intended for consumption +D. The reporting tool used to visualize the data + +**My Answer:** C + +**Result:** ✅ Correct + +--- + +## Question 2 + +**Question:** Which statement is correct? + +A. A Data Lake is always Bronze and a Data Warehouse is always Gold +B. Bronze, Silver, and Gold describe data maturity, while Data Lake and Data Warehouse describe architecture/storage approaches +C. Azure Synapse is another name for the Gold layer +D. A Lakehouse is simply a Data Warehouse hosted in Azure + +**My Answer:** B + +**Result:** ✅ Correct + +--- + +## Question 3 + +**Question:** What is the main difference between ETL and ELT? + +A. ETL is used only in the cloud +B. ELT does not transform data +C. ETL transforms before loading, while ELT loads before transforming +D. ETL uses Fact Tables while ELT uses Dimension Tables + +**My Answer:** C + +**Result:** ✅ Correct + +--- + +## Question 4 + +**Question:** A table contains one row for every individual video viewing session. What does this describe? + +A. The Semantic Layer +B. The Grain +C. The Data Lineage +D. The Dimension + +**My Answer:** B + +**Result:** ✅ Correct + +--- + +## Question 5 + +**Question:** Which is most likely a Fact Table? + +A. `dim_platform` +B. `dim_date` +C. `fact_viewing` +D. `dim_content` + +**My Answer:** C + +**Result:** ✅ Correct + +--- + +## Question 6 + +**Question:** Which is most likely a Dimension Table? + +A. A table containing one row per video viewing event +B. A table containing content title, genre, series, and language +C. A table containing total watch seconds per session +D. A table containing raw API events + +**My Answer:** B + +**Result:** ✅ Correct + +--- + +## Question 7 + +**Question:** In a Star Schema: + +A. Every dimension connects to another dimension before reaching the Fact Table +B. The Fact Table is normally at the center and dimensions connect directly to it +C. There are no Fact Tables +D. All data is stored in one large table + +**My Answer:** B + +**Result:** ✅ Correct + +--- + +## Question 8 + +**Question:** Compared with a Star Schema, a Snowflake Schema generally: + +A. Has fewer tables and fewer joins +B. Stores only raw data +C. Further normalizes Dimension Tables into related tables +D. Eliminates the need for a Fact Table + +**My Answer:** C + +**Result:** ✅ Correct + +--- + +## Question 9 + +**Question:** What is the main purpose of a Semantic Layer? + +A. Store raw data +B. Replace the Data Warehouse +C. Provide consistent business definitions and reusable metrics +D. Manage SSL certificates + +**My Answer:** C + +**Result:** ✅ Correct + +--- + +## Question 10 + +**Question:** What does Data Lineage primarily help answer? + +A. How attractive a dashboard looks +B. Where data came from, how it changed, and where it is used +C. Which database technology is cheapest +D. How many Dimension Tables should exist + +**My Answer:** B + +**Result:** ✅ Correct + +--- + +# Section B - Short Response + +## Question 11 + +**Question:** Explain the difference between Bronze and Silver. + +### My Answer + +> Bronze is the raw data. Silver is a cleaned up data but not yet fully ready for business use. + +### Result + +✅ **Correct** + +### Key Point + +[[Bronze Layer]] contains raw or minimally processed source data. + +[[Silver Layer]] contains cleaned, validated, standardized, or enriched data that is more trustworthy but not necessarily structured for final business consumption. + +--- + +## Question 12 + +**Question:** Explain the difference between a Data Lake and a Data Warehouse. + +### My Answer + +> Data lake collects all raw data from every source. While the Data Warehouse has the data more organized. Hesitant to even say that data is transformed for the warehouse in ETL. + +### Result + +🟡 **Partially Correct** + +### Correction + +A [[Data Lake]] can store large amounts of **raw and processed data** in many formats. It is not limited to raw data. + +A [[Data Warehouse]] contains highly structured data designed primarily for analytics and reporting. + +The important distinction is: + +> **Data Lake = flexible storage for diverse data.** + +> **Data Warehouse = structured analytical system optimized for querying and reporting.** + +Data may be transformed before entering a Data Warehouse in an ETL architecture, but with modern ELT architectures, data can also be loaded first and transformed afterward. + +### What I Need to Remember + +Do not automatically think: + +> Data Lake = Raw +> Data Warehouse = Transformed + +Instead: + +> **Lake vs Warehouse = architecture/storage approach** + +> **Bronze vs Silver vs Gold = data maturity** + +--- + +## Question 13 + +**Question:** What does this question mean? + +> "What is the grain of this Fact Table?" + +### My Answer + +> It is a question asking what a single row in this Fact Table means. + +### Result + +✅ **Correct** + +### Key Point + +[[Grain]] defines exactly what **one row represents** in a [[Fact Table]]. + +Example: + +> One row = one viewing session for one piece of content by one user. + +--- + +## Question 14 + +**Question:** Explain the relationship between a Fact Table and a Dimension Table. + +### My Answer + +> Dimension are data that cannot be aggregated and provide context to those tables that can be aggregated. The later are facts table. + +### Result + +🟡 **Partially Correct** + +### Correction + +A [[Fact Table]] records **business events or measurable observations** and commonly contains [[Measure|Measures]] such as revenue, watch seconds, quantity, or impressions. + +A [[Dimension Table]] provides **descriptive context** about those events. + +For example: + +`fact_viewing` tells us: + +> A viewing event happened for 125 seconds. + +Dimensions tell us: + +> **Who** watched? +> **What** was watched? +> **When**? +> **Where or how**? + +The distinction is not simply that Facts can be aggregated and Dimensions cannot. Dimension Tables can contain numeric attributes too. + +The important distinction is their **role in the analytical model**. + +### What I Need to Remember + +> **Fact = What happened?** + +> **Dimension = Describe what happened.** + +--- + +## Question 15 + +**Question:** Why might a business choose a Star Schema instead of a Snowflake Schema for Power BI? + +### My Answer + +> Not sure how to answer this, but snowflake would be used to respond specific business questions. Often snowflake will provide deeper analysis since the database is normalized. + +### Result + +❌ **Incorrect** + +### Correction + +A business might choose a [[Star Schema]] instead of a [[Snowflake Schema]] for Power BI because a Star Schema generally has: + +- Fewer tables + +- Fewer joins + +- Simpler relationships + +- Easier navigation for analysts + +- A model that is easier for BI tools and users to understand + + +A Snowflake Schema is more normalized, but this does **not** mean it provides deeper analysis. + +Both models can potentially answer the same business questions. + +The primary trade-off is: + +> **Star Schema = simplicity** + +> **Snowflake Schema = normalization and reduced duplication** + +### What I Need to Remember + +Normalization does not automatically mean better or deeper analytics. + +For BI: + +> **Simple models are often preferable unless additional complexity solves a real problem.** + +--- + +## Question 16 + +**Question:** Why can two Power BI dashboards show different values for the same KPI even if neither report contains a technical error? + +### My Answer + +> The semantic layer must be reviewed to ensure that the metrics in all reports have the same definitions. + +### Result + +✅ **Correct** + +### Key Point + +Two reports can be technically correct while using different business definitions. + +A governed [[Semantic Layer]] helps ensure that a KPI is defined once and reused consistently across reports. + +--- + +## Question 17 + +**Question:** Give one example of how Data Lineage could help during a vendor migration. + +### My Answer + +> Not sure.. all I can think of is the impact on a project that this could have since it defines the scope better. + +### Result + +🟡 **Partially Correct** + +### Correction + +This answer was heading in the right direction. + +During a vendor migration, [[Data Lineage]] helps identify **everything downstream that depends on the vendor's data**. + +For example: + +```text +JW Player +↓ +Viewing Pipeline +↓ +fact_viewing +↓ +Total Watch Hours +↓ +Semantic Layer +↓ +Executive Dashboard +↓ +Content Performance Dashboard +``` + +Before replacing JW Player, Data Lineage helps determine which: + +- Pipelines + +- Transformations + +- Tables + +- Metrics + +- Reports + + +will be affected. + +This does help define the true project scope through [[Impact Analysis]]. + +### What I Need to Remember + +The key CTO question is: + +> **"If we change this system, what downstream systems, data, metrics, and reports will be affected?"** + +--- + +# Section C - Scenario Questions + +## Question 18 - Audience vs Sessions + +**Question:** + +An analyst presents: + +> "Our audience increased by 30%." + +You ask how audience was calculated. + +They answer: + +> "We counted the number of rows in `fact_viewing`. One row represents one viewing session." + +What is wrong with the analyst's statement, and what should the metric probably be called? + +### My Answer + +> A viewing doesn't represent an audience. We must first agree to what is an audience. A multiple viewing could be linked to one person. + +### Result + +✅ **Correct** + +### Key Point + +If the [[Grain]] is one row per viewing session, counting rows measures **viewing sessions**, not necessarily audience. + +One person can generate multiple viewing sessions. + +The metric should likely be called: + +> **Viewing Sessions** + +A separate governed definition is needed for: + +> **Audience** or **Unique Viewers** + +--- + +## Question 19 - Competing KPI Definitions + +**Question:** + +Marketing defines: + +> Active User = anyone who opened the website + +Product defines: + +> Active User = anyone who watched at least one video + +Both teams publish dashboards showing "Active Users." + +What architectural or governance problem do you see, and what concept from the lessons could help solve it? + +### My Answer + +> A semantic layer where the definition of items is common to all. Basically, create a governed shared business definition. + +### Result + +✅ **Correct** + +### Key Point + +The organization has competing definitions of the same business metric. + +A governed [[Semantic Layer]], supported by [[Data Governance]], should establish a shared definition of **Active User** that can be consistently reused. + +--- + +## Question 20 - API Change + +**Question:** + +A vendor announces that the field `watch_time_ms` will be removed from its API. + +Before approving the technical change, what should you ask the team to determine? + +Give 2 or 3 things. + +### My Answer + +> Don't know the answer.. has to do with data lineage but I didn't read it properly. + +### Result + +❌ **Incorrect** + +### Correction + +This is a [[Data Lineage]] and [[Impact Analysis]] question. + +I should ask the team to determine: + +1. **Where is this field used downstream?** + + +Which pipelines, Bronze/Silver/Gold transformations, [[Fact Table|Fact Tables]], or other datasets depend on it? + +2. **Which metrics depend on it?** + + +For example: + +```text +watch_time_ms +↓ +watch_seconds +↓ +Total Watch Hours +``` + +3. **Which reports or business processes will be affected?** + + +For example: + +```text +Total Watch Hours +↓ +Executive Dashboard +↓ +Content Performance Report +``` + +The goal is to understand the complete impact **before** treating it as simply an API change. + +### What I Need to Remember + +When a source changes: + +> **Trace forward.** + +```text +Source +↓ +Pipeline +↓ +Data Model +↓ +Metric +↓ +Report +``` + +When a KPI looks wrong: + +> **Trace backward.** + +```text +Report +↑ +Metric +↑ +Data Model +↑ +Pipeline +↑ +Source +``` + +This is [[Data Lineage]]. + +--- + +# Section D - CTO Judgment + +## Question 21 + +**Question:** A data architect says: + +> "We should move to a Lakehouse because it's the modern architecture." + +What is the best response? + +A. "Yes, let's start immediately." +B. "No, Data Warehouses are better." +C. "What problem are we trying to solve that our current architecture cannot solve?" +D. "Let's ask Power BI which architecture it prefers." + +**My Answer:** C + +**Result:** ✅ Correct + +### Key Point + +Architecture should solve a problem. + +Do not adopt technology simply because it is newer. + +--- + +## Question 22 + +**Question:** An architect wants to split one Dimension Table into six smaller tables because: + +> "It's more normalized and technically cleaner." + +What is the strongest CTO-style response? + +A. "Normalization is always better, so let's do it." +B. "How does the additional complexity improve governance, maintainability, performance, or business outcomes?" +C. "Snowflake Schemas are bad." +D. "Let the developers decide without discussing it." + +**My Answer:** B + +**Result:** ✅ Correct + +### Key Point + +Architectural complexity should provide measurable value. + +Technical elegance alone is not sufficient justification. + +--- + +## Question 23 + +**Question:** A dashboard shows a sudden 40% drop in watch hours. + +What should you investigate before concluding that audience behavior changed? + +A. Whether the dashboard colors changed +B. Whether the source, pipeline, transformations, or upstream data quality changed +C. Whether the CEO likes the metric +D. Whether a new Dimension Table should be created + +**My Answer:** B + +**Result:** ✅ Correct + +### Key Point + +A technically correct dashboard can display incorrect or incomplete business information if something upstream has failed. + +Use [[Data Lineage]] to trace backward and investigate [[Data Quality]]. + +--- + +## Question 24 + +**Question:** Two reports disagree on platform performance because each analyst created their own mapping of Roku, Apple TV, and Android TV. + +What is the strongest long-term solution? + +A. Let each analyst keep their own definition +B. Pick whichever dashboard has the higher number +C. Create a governed shared platform dimension or shared business definition +D. Merge all the reports into one giant spreadsheet + +**My Answer:** C + +**Result:** ✅ Correct + +### Key Point + +Shared business concepts should be governed centrally rather than recreated independently in individual reports. + +This connects: + +- [[Dimension Table]] + +- [[Semantic Layer]] + +- [[Data Governance]] + + +--- + +# Bonus - Question 25 + +**Question:** + +In one sentence, explain the entire path from source system to executive dashboard using as many of these concepts as you can: + +`ETL/ELT`, `Bronze`, `Silver`, `Gold`, `Fact Table`, `Dimension Table`, `Star Schema`, `Semantic Layer`, `Power BI`, `Data Lineage` + +### My Answer + +> Not answered. + +### Result + +➖ **No penalty** + +--- + +# Final Result + +**Score: 20.5 / 24** + +**Percentage: 85%** + +## What I Understand Well + +I have a strong understanding of the overall data architecture: + +```text +Source Systems +↓ +[[ETL vs ELT]] +↓ +[[Medallion Architecture]] +↓ +Bronze → Silver → Gold +↓ +Analytical Data Model +↓ +[[Fact Table]] + [[Dimension Table]] +↓ +[[Star Schema]] / [[Snowflake Schema]] +↓ +[[Semantic Layer]] +↓ +Power BI +↓ +Business Decisions +``` + +I also understand the CTO-level principle of asking: + +> **"What problem are we trying to solve?"** + +before choosing an architecture or technology. + +## What I Need to Reinforce + +### 1. Data Lake vs Data Warehouse + +Remember: + +> **Data Lake vs Data Warehouse = architecture/storage** + +> **Bronze vs Silver vs Gold = data maturity** + +A Data Lake is not automatically Bronze. + +A Data Warehouse is not automatically Gold. + +--- + +### 2. Fact vs Dimension + +Remember: + +> **Fact = What happened?** + +> **Dimension = Describe what happened.** + +Do not define the difference only as: + +> Facts contain numbers; dimensions do not. + +--- + +### 3. Star vs Snowflake + +Remember: + +```text +[[Star Schema]] +Simple +Fewer joins +BI-friendly +Some duplication is acceptable + +vs. + +[[Snowflake Schema]] +More normalized +More tables +More joins +Less duplication +Potentially more complexity +``` + +Normalization does **not** mean deeper analysis. + +Both can answer complex business questions. + +--- + +### 4. Data Lineage + +Remember two directions: + +**Something is wrong with a KPI?** + +```text +Dashboard +↑ +Metric +↑ +Model +↑ +Gold +↑ +Silver +↑ +Bronze +↑ +Source + +TRACE BACKWARD +``` + +**A source or vendor is changing?** + +```text +Source +↓ +Pipeline +↓ +Model +↓ +Metric +↓ +Reports + +TRACE FORWARD +``` + +The CTO-level question is: + +> **"If this changes, what downstream systems and decisions are affected?"** + +That is [[Impact Analysis]]. \ No newline at end of file