Insights

Notes on data engineering

Working notes on dbt, pipelines, warehouses and the failure modes that show up once a data platform has been running for a while.

No articles match your search.

Isometric illustration of data pipelines connecting databases with a security shield and growth chart
Data Engineering
March 15, 2026 · 15 min read
Building Resilient Data Pipelines at Scale
Idempotency, schema evolution, dead-letter records and backoff: the patterns that keep a data pipeline trustworthy once it is carrying real volume, and a worked example of the watermark bug that silently drops rows.
Diagram contrasting data flowing into a warehouse via traditional ETL with data flowing out to operational tools via reverse ETL
Data Engineering
June 1, 2026 · 7 min read
Reverse ETL vs. Traditional ETL: Choosing the Right Data Movement Pattern
Traditional ETL moves data into your warehouse. Reverse ETL moves it back out to the tools your teams actually use. Here's how to decide which direction your next pipeline needs to run.
Side-by-side comparison of a version-controlled dbt project structure and a database stored procedure editor
Data Engineering
June 8, 2026 · 7 min read
dbt vs. Stored Procedures: Modern Data Transformation Compared
Stored procedures have transformed data inside the warehouse for decades. dbt applies software engineering discipline to the same job. The difference shows up in testing, versioning, and how fast your team can move.
Architecture diagram showing a legacy on-premise ETL server transitioning to a cloud-native managed pipeline stack
Data Engineering
June 15, 2026 · 8 min read
Legacy ETL vs. Cloud-Native Data Pipelines: What Changes When You Modernize
Informatica and SSIS built the last generation of enterprise data platforms. Here's what actually changes when you move to a cloud-native stack, and what doesn't.