Analytics engineering
We build and maintain dbt projects: staging, intermediate, and mart layers, with tests, documentation, and CI/CD built in from day one.
dbt model development is the practice of writing modular SQL transformations, organised into staging, intermediate, and mart layers, with automated tests and generated documentation, so an analytics team can trust and trace every metric back to its source.
The problem
Most dbt projects start clean. They don't stay that way.
A project starts with eight models and clear naming conventions. It grows to two hundred, with a couple of people left who remember why int_orders_v2_final exists, CI that passes while the numbers are wrong, and a warehouse bill that climbs every quarter.
This is a common failure mode in growing dbt projects, not a rare one, and it's fixable with the same layering discipline that would have prevented it in the first place.
Our approach
Staging models map one-to-one with source tables: renamed, cast, and nothing else. Intermediate models hold the joins and the business logic, built to be reused and never queried directly by a BI tool. Marts are the final, documented tables your dashboards and stakeholders actually read.
Every model gets a primary-key test and a not-null test at minimum. Every project gets dbt docs generation and a CI pipeline that runs dbt build on each pull request. This isn't a stylistic preference. It's what stops the failure modes above from coming back.
A dbt project rarely lives alone. If yours needs to sit inside a broader ingestion and warehousing setup, see Data Platform Architecture.
Our methodology
01
stg_* One model per source table. Raw data cleaned, renamed, and cast to the right types. No joins, no business logic. This layer is what makes everything downstream stable and refactorable.
02
int_* Joins, deduplication, and business logic live here, with ref() dependencies declared explicitly. This is where data quality issues get caught before they reach analysts.
03
fct_*dim_* Analytics-ready fact and dimension tables, designed for direct BI tool consumption, with metric definitions aligned to how the business actually measures performance.
What we deliver
Greenfield setup: naming conventions, layering strategy, and a dbt_project.yml that still makes sense in year two.
Existing projects: untangling circular refs, adding coverage to untested load-bearing models, and fixing slow builds caused by missing incremental strategies.
Generic and singular tests, full dbt docs generation, and source freshness checks on the tables everything else depends on.
dbt build on every pull request, state-deferred runs so only affected models rebuild, and a promotion path into production.
Incremental materialisation, partitioning and clustering strategy, and removing the work a build repeats on every single run.
Schema migration, versioned DDL, and the Lambda, ECS, S3, and Terraform layer a dbt project needs in order to actually run in production.
Where we work
Production dbt.
Schema and platform layer, stated with its real boundary.
What we don't claim: senior-level warehouse operations on Snowflake. Credit management, clustering strategy, RBAC hierarchy design and Snowpipe are real, current work for someone else. We would rather say so up front than have a project find out midway through.
Credibility, not claims
models/staging/stg_orders.sql
-- One model per source table: rename, cast, nothing else.
with source as (
select * from {{ source('ecommerce', 'orders') }}
),
renamed as (
select
order_id,
customer_id,
cast(created_at as timestamp) as order_created_at,
status as order_status
from source
)
select * from renamedmodels/intermediate/int_order_items.sql
-- Joins and business logic, on ref() only, never raw tables.
with orders as (
select * from {{ ref('stg_orders') }}
),
order_items as (
select * from {{ ref('stg_order_items') }}
),
joined as (
select
o.order_id,
o.customer_id,
o.order_created_at,
oi.product_id,
oi.quantity,
oi.unit_price,
oi.quantity * oi.unit_price as line_total
from orders o
join order_items oi using (order_id)
)
select * from joinedmodels/staging/schema.yml
version: 2
models:
- name: stg_orders
description: "Cleaned and renamed orders from the ecommerce source."
columns:
- name: order_id
tests:
- not_null
- unique
- name: order_status
tests:
- accepted_values:
values: ['pending', 'completed', 'cancelled', 'returned']Before & after
select
o.id as order_id,
o.customer_id,
c.email,
o.total / 100.0 as total_usd,
case when o.status = 'A' then 'active'
when o.status = 'C' then 'cancelled'
else 'unknown' end as status,
o.created_at
from raw.orders o
left join raw.customers c
on o.customer_id = c.id
where o.deleted_at is null-- staging/stg_orders.sql
select
id as order_id,
customer_id,
total / 100.0 as total_usd,
status_code as status
from {{ source('raw', 'orders') }}
where deleted_at is null
-- marts/fct_orders.sql
select
o.order_id,
o.customer_id,
c.email,
o.total_usd,
o.status
from {{ ref('stg_orders') }} o
join {{ ref('stg_customers') }} c
using (customer_id)How we work
We map every model, source, and test that exists today, or the sources you need to model from scratch.
Staging, intermediate, and mart boundaries, plus naming standards your team will actually follow.
Models and tests are built and reviewed layer by layer, not all at once at the end.
dbt build in your existing CI, with state-deferred runs once the project is large enough to benefit.
A working session with your analytics engineers, so the project doesn't depend on us to maintain.
Why layering matters
| Ad-hoc SQL in the warehouse | dbt without conventions | Layered dbt project | |
|---|---|---|---|
| Lineage | Undocumented, lives in people's heads | Partially traceable through ref() | Full DAG, generated from the project |
| Test coverage | Manual spot checks, after the fact | A few tests, on whichever models someone remembered | Not-null and uniqueness on every key, enforced in CI |
| Changing a definition | Find and edit every query that uses it | Edit several models and hope nothing downstream breaks | Change one staging model; everything downstream follows |
| Onboarding an engineer | Ask whoever wrote it | Read the models and infer the rules | Read the conventions, then the docs the project generates |
Ways to work together
Fixed-price, five business days
EUR 2,500 fixed, 30% deposit before work starts
For larger or ongoing work
Rate quoted against scope after a first call
Common questions
A dbt model is a SQL (or Python) file that defines a transformation as a select statement. dbt compiles it, manages the dependency graph between models via ref(), and materializes it in your warehouse as a table or view.
dbt Core is the open-source CLI you run yourself, in your own CI/CD. dbt Cloud is a hosted service that adds scheduling, an IDE, and a managed environment on top of the same underlying engine. We work with both, depending on what a project already has in place.
A greenfield build is typically four to eight weeks depending on the number of source systems and the complexity of the business logic. A rescue of an existing project, auditing and restructuring what's already there, is usually two to four weeks. The fixed-price audit sprint above is five days and answers the question of where to start.
Production dbt work on Redshift, and Snowflake at the schema and platform layer: schema migration, versioned DDL, and hybrid tables. See the warehouse section above for exactly what that does and doesn't cover.
Knowledge transfer is built into every engagement: documentation generated from the project itself, plus a walkthrough session so your team can maintain what's delivered without us in the room. A standalone training engagement can be scoped on request.
Tell us what's broken or what you're building, and we'll take it from there.
Ask VividLogic
AI assistant · answers about our services
AI-generated answers may be imperfect. For quotes or scoping, use /contact.