This dbt models is designed for teams moving from research to an accountable delivery decision. Use the ELT workflow planning guide, warehouse modeling guide and data quality guide for adjacent scope, implementation and operating questions. The practical standard here is evidence: named owners, explicit boundaries, representative tests and a route to stop or correct the system when assumptions fail.
dbt models turn SQL or Python transformation logic into named, dependency-aware analytics assets. The official dbt introduction describes transformation workflows with modular code, testing, documentation and collaboration. Useful projects still depend on warehouse design, source reliability and operating ownership. This guide complements the Edilec data lineage guide by focusing on decisions that keep a dbt dependency graph understandable and safe to change.
Choose model boundaries around meaning and change
Start from grain and responsibility. A staging model should rename, cast and lightly standardize one source without hiding business policy. Intermediate models can isolate reusable joins or transformations. Marts should expose stable business entities, events and measures for a declared audience. Dbt’s project-structure guide recommends staged layers, but names and folders are useful only when reviewers can predict where logic belongs. Keep each model conceptually narrow and select explicit columns at stable interfaces. Avoid one enormous query that mixes cleanup, identity resolution, financial rules and presentation, as well as dozens of tiny pass-through models that add no meaning. Document grain, keys, owner, purpose and consumers. When two valid definitions serve different decisions, name them separately rather than forcing misleading reuse.
Select materialization from behavior and evidence
Views are simple and current but can push repeated work onto consumers. Tables provide predictable reads but require refresh and storage. Incremental models reduce processed data but introduce state, late-arriving records, updates and full-refresh recovery. Ephemeral models inline logic and can make compiled queries harder to understand when overused. Official incremental-model guidance requires filtering rows during incremental runs and configuring how updates are handled. Write down unique key, lookback, mutation behavior, schema change, backfill and rebuild time before choosing incremental. Test with changed and late data, not only inserts. Measure warehouse runtime and downstream latency under representative use. Materialization is an operational contract; changing it can alter freshness, cost, locking and failure behavior even when output columns remain the same.
| Layer | Purpose | Good boundary | Avoid |
|---|---|---|---|
| Source | Declare loaded data | Freshness and loader metadata | Business logic |
| Staging | Standardize one source | Rename, cast and basic cleanup | Cross-domain joins |
| Intermediate | Reuse transformation | Identity or event logic | Unowned dumping ground |
| Mart | Serve a decision | Stable entity and measures | Mixed incompatible grains |
| Exposure | Name downstream use | Owner and dependency | Invisible consumption |
Test invariants that protect decisions
Use generic tests for not-null, unique, accepted values and relationships, then add singular tests for business invariants and reconciliation. Dbt data tests are assertions over warehouse data; a failing query returns rows that violate the assertion. Place tests where a defect can first be identified and again at consequential interfaces. Severity and timing should follow impact: duplicate payment identifiers may block release, while a small optional-description gap may warn. Test source freshness and schema, key preservation, aggregation totals, eligibility rules and historical behavior. Retain enough failed rows to diagnose without exposing sensitive data. Assign every important failure an owner and response. A large count of tests is not coverage if all assertions verify easy null checks while the revenue allocation rule remains untested.
Use contracts and documentation deliberately
A model contract can enforce column names and data types, and constraints where supported. Dbt’s model-contract documentation warns that contract changes can be breaking and require planning. Apply contracts to stable, widely consumed interfaces rather than every exploratory model. Pair structural enforcement with semantic documentation: grain, purpose, owner, refresh, calculations, sensitivity, known limitations and examples. Document sources with database identifiers, freshness expectations and loading behavior using source properties. Generate docs in CI or deployment and keep them accessible to consumers. Documentation is weakest when it paraphrases column names; it is strongest when it settles ambiguity and points to authority. Version or create compatibility models for material interface changes, notify consumers and define retirement.
Build a safe state-aware release path
Develop against representative but protected data and isolate schemas per engineer or branch. Run formatting, compilation, unit checks where used, data tests and selected builds in CI. Use dependency state to limit work only after understanding deferred environments and changed upstreams; periodically run broad builds to expose hidden coupling. Review compiled SQL and query plans for expensive changes. Deploy through versioned jobs and credentials with least privilege. Backfill in controlled windows, reconcile before switching consumers and preserve rollback or a known-good artifact. Separate code success from data success: a query can compile yet produce semantically wrong rows. Record invocation, code revision, environment and test results. For critical marts, canary downstream dashboards or extracts before full promotion and coordinate breaking changes with consumers.
| Change | Required evidence | Release control | Recovery |
|---|---|---|---|
| New column | Definition and sensitivity | Docs and test | Remove before consumers |
| Type change | Compatibility impact | Contract and version | Compatibility model |
| Incremental logic | Late/update scenarios | Rebuild comparison | Full refresh |
| Metric rule | Business approval | Reconciliation and notice | Prior version |
| Source migration | Parallel result | Dual run and cutover | Restore old source |
Operate dbt models as production data products
Monitor source freshness, run duration, queue time, model status, test failures, row and volume anomalies, cost and downstream availability. Correlate an incident to source load, invocation and code version. Define service objectives around when usable data reaches a decision, not simply when the dbt job ends. Retry only transient work; quarantine or stop when repeating a semantic defect would spread harm. Keep runbooks for failed source, partial build, backfill, contract break, warehouse limit and credential rotation. Review ownership and unused models, remove orphaned exposures and expire temporary exceptions. Measure cost by product or workload where possible. A green scheduler is insufficient if consumers receive stale or unreconciled data. Operating reviews should prioritize root causes and reduce repeated manual intervention.
Work through an incremental orders mart
Suppose an orders mart supports daily fulfillment and monthly finance. Define one row per order line, stable source identifiers, event and business dates, currency policy, cancellation states and owners. Stage source tables separately, resolve customer and product keys in intermediate models, then publish operational and financial marts with deliberately different freshness. An incremental model uses updated_at plus a lookback because refunds can arrive late; a reconciliation test compares monthly totals with the finance source. Contracts protect published keys and amounts. During rollout, a late currency correction exposes that the unique key omitted legal entity, so the team fixes the grain, runs a bounded rebuild and compares downstream reports before promotion. The runbook captures this scenario. The graph now explains transformations, but ownership, tests and reconciliation make it trustworthy.
Review the graph and rehearse recovery
Periodically review the project as a dependency system. Identify excessive fan-in or fan-out, duplicated logic, unclear ownership, unstable sources, missing exposures, long critical paths and high warehouse cost. Trace a published metric backward through grain, joins, filters and freshness, then trace a source change forward to consumers. Compare documented ownership with incident response. Review permissions for development, deployment, production and generated documentation. Check indirect consumers before removing models. Then exercise recovery for critical products using late data, accidental refresh, broken contract, duplicate incrementals, warehouse limits or credential failure. Ask the owner to detect, contain, communicate, repair, reconcile and document with existing telemetry and runbooks. Measure time until consumers receive trustworthy data, not job restart. Verify backfills preserve valid history and cost controls. Convert findings into tests, alerts, optimization or clearer responsibility. Reliable dbt practice comes from repeatedly proving the graph can change and recover without leaving consumers to discover silent errors.
Define ownership at model, domain and platform levels. Model owners answer semantic and test failures; domain owners settle business definitions and source precedence; platform owners manage execution, credentials, warehouse limits and shared tooling. An incident commander coordinates failures spanning these boundaries. Publish contacts in model metadata and route alerts according to consequence. Review temporary models, disabled tests, warnings and exceptions with expiry dates. Require an owner before a model becomes a supported interface and a retirement plan when ownership ends. Clear responsibility keeps decentralized analytics development fast without leaving critical marts as communal assets that everyone queries but nobody can safely change.
Key takeaways
- Keep temporary exceptions, disabled tests and experimental models visible with owners and expiry dates so production debt cannot disappear inside a successful schedule.
- Model boundaries should clarify grain, meaning and ownership.
- Treat materialization as an operating decision.
- Test business invariants and reconciliation, not just easy schema properties.
- Use contracts on stable interfaces and accompany them with semantic documentation.
- Release with state-aware evidence, broad periodic builds and tested recovery.
Frequently asked questions
How many dbt models should a project have?
There is no useful target count. Create a model when it establishes a meaningful boundary, reusable transformation or stable interface. Avoid both giant multi-purpose queries and pass-through layers that add no meaning.
Should every model be incremental?
No. Incremental logic adds state and recovery complexity. Use it when full processing misses a measured cost or latency objective, and only after defining keys, updates, late data, backfill and full refresh.
Do dbt tests replace source and application controls?
No. They can detect warehouse assertions after data arrives. Source validation, application constraints, reconciliation, access control, observability and business review remain necessary.
Conclusion
dbt models are dependable when their grain, meaning, materialization, tests, contracts and owners are explicit. Build a graph that reviewers can reason about, release changes with representative data and compatibility evidence, and operate marts around consumer decisions. That turns transformation code into a maintainable analytics product rather than a collection of scheduled queries.