[Gold Layer]
Canonical Marts on a Medallion Lake
My part of the company's data lake: canonical sales marts and the audit for moving dashboards off the legacy database.
The data team was building an AWS lake (landing → bronze → silver → gold with Glue, Athena and Terraform) to replace the legacy MySQL as the source for dashboards.
Contributor. I built the canonical front-end and back-end sales marts — first as views, then as Iceberg tables — and wrote the migration feasibility audit. The lake itself is team work.
Dashboards couldn't move to the lake while no mart reproduced the numbers people already trusted.
01 · Canonical names
fct_ / dim_ / met_ naming, daily partitions, and every money column in two currencies.
02 · Consume, don't recompute
Tax and fees come as fields from the source; recomputing them in the mart drifted from finance.
03 · Pruning at both ends
Filter on the mart's date and on the source order date inside the join, or the engine scans every partition.
04 · Parity audit
Each legacy dashboard mapped to the marts it would need, with gaps measured — an offer-name parse that missed part of the revenue, a placeholder unit cost.
what you're seeing
Four layers of a lake. Click one to see what the same three orders look like there: raw JSON, typed Parquet, cleaned and deduplicated rows, and a gold mart row. The IaC tab shows which infrastructure modules touch each layer.try this
- 1.Go from Landing to Silver and spot the duplicate order disappearing and the timestamp moving to UTC.
- 2.In Gold, switch USD/BRL: every money column exists in both currencies.
- 3.Read the partition-pruning note — it's the most common reason a mart query scans everything.
real vs simplified
Layer conventions and the gold rules are the real ones. Orders and module names are fictional; the lake itself is team work, my part was the marts.! brokeRefunds were counted twice: one gateway emits a 'cancellation' event for money that was already refunded.
→ changedDocumented which event families overlap and excluded the duplicate bucket.