Data Mesh Contracts: What One Domain Can Rely On From Another

Picture a team that builds a daily city report for a ride-hailing app: trips requested, trips completed and revenue, one row per city per day. Trips come from the Marketplace team's tables. Revenue comes from the Payments team's card charges. One week Payments adds a column and starts sending one market's amounts in pence instead of pounds. Every job succeeds. The report's revenue for that city is a hundred times too high, and nobody can say whose job it was to notice.

Each team's pipeline was correct by its own definition. What failed was the agreement between them, which existed only as a shared assumption, a wiki page or a message thread. A data mesh gives each domain its own data; it does not, by itself, write down what a consumer in another domain may rely on.

This article separates three things that often get called "doing data mesh" (organising by domain, declaring owners and isolating access), compares the common ways teams write the cross-domain agreement down, and walks through one way to make that agreement checkable when the pipeline runs. It does not cover writing access grants in depth, and a contract cannot catch a wrong value that still looks plausible.

Three things that get called a data mesh

Zhamak Dehghani's data mesh principles are domain ownership, data as a product, a self-serve platform and federated computational governance. In practice, three different properties hide inside "domain ownership", and each is established by something different:

None of these implies the next. A catalog per domain is a namespace, not a permission boundary until someone writes grants. An owner field in a file is a statement, not accepted accountability. And none of the three says what a consumer can expect from the data itself: which columns exist, what types they have, which values are allowed, and what happens to a row that breaks those expectations. That fourth thing is the interface, and it is the subject of the rest of this article.

Why the interface is where it breaks

Inside one team, a schema change and the code that reads it are usually reviewed together. Across domains, the producer changes its table on its own schedule and the consumer finds out from its output. The producer often cannot see who reads its tables, and the consumer has no written promise to point to. Quality rules, where they exist, tend to sit in the consumer's code, so each consumer re-implements its own idea of a valid charge.

Writing the interface down helps only if three questions have answers: who may change it, how a change gets reviewed, and what happens at run time when the data does not match it.

The ways teams write the interface down

Several approaches work, and many teams combine them. Each is a reasonable choice in the right setting:

A few notes on each. In Databricks, a catalog per business unit or domain is a common Unity Catalog layout, and it gives grants a natural place to attach. dbt's model contracts check column names and types before a model builds, and dbt's model access settings let a project mark models as public to other projects; the dbt docs note that content checks belong to data tests and that constraint enforcement varies by platform. The Open Data Contract Standard (ODCS) defines a platform-neutral YAML contract covering schema, quality, team, roles and service levels; checking data against it is a job for a tool such as the open-source Data Contract CLI. Data Mesh Manager adds a marketplace of data products and their contracts, and integrates that CLI for checks. Platform-native rules such as Databricks pipeline expectations warn about, drop or stop on failing records inside that platform's pipelines.

The question these leave partly open is narrower than "how do I do data mesh". It is: can the producer's promise and the consumer's dependency sit in the same reviewable file, be checked on every run with failing rows kept for inspection, and travel unchanged if the estate spans more than one platform? That is the gap LakeLogic is built for.

A worked example: RideFlow's city report

RideFlow, a fictional ride-hailing business, is the reference project we use to show this. Its city report is the shared marketplace_health product: for each city and day, trips requested, completed and cancelled, active drivers and riders, and gross revenue. It reads trip tables owned by the Marketplace domain and card charges owned by Payments. The question it answers is the one from the opening: how do demand, completion and revenue vary by city and day?

Around that product sits a full estate: 158 contracts across seven domains (Marketplace, Payments, Operations, Marketing, Business, Reference and Shared) and Bronze, Silver and Gold layers, all synthetic data. The Databricks edition puts each domain in its own Unity Catalog catalog. The Microsoft Fabric and Snowflake editions state in their READMEs that they run the same contracts and differ only in the platform layer.

  Marketplace domain                  Payments domain
  rideflow_dev_marketplace            rideflow_dev_payments
    silver_rideflow_trip_requests       silver_stripe_charges
    silver_rideflow_trip_completed             │
    silver_rideflow_trip_cancellations         │
             │                                 │
             └──────────────┬──────────────────┘
                            ▼
              Shared domain  rideflow_dev_shared
                gold_mart_marketplace_health
                one row per (kpi_date, city_code)

The producer's side: what a trip row promises

The Marketplace domain's Silver contract for completed trips declares each field, its type, whether it is required, and row rules. A shortened excerpt from the repository:

info:
  domain: marketplace
  system: rideflow

model:
  fields:
  - name: trip_id
    type: string
    required: true
  - name: fare_amount
    type: decimal(10,2)
  # ...

primary_key:
- trip_id

quality:
  row_rules:
  - name: fare_amount_range_min
    sql: fare_amount IS NULL OR fare_amount >= 0
  # ...

Ownership is declared one level up, in the domain's _domain.yaml, together with service expectations and retention. In the reference project, Payments names "Finance Engineering" while three domains, including Marketplace, name "Platform Data Engineering". That is worth noticing: the file records who is named, and whether that team has accepted the role is a conversation, not a field.

The consumer's side: a dependency you can read

The shared product's contract names every table it reads across the boundary, by catalog:

primary_key: [kpi_date, city_code]

links:
  - name: mp_trips
    type: table
    path: "table:`{domain_catalog_prefix}_marketplace`.silver.silver_rideflow_trip_completed"
  - name: pay_charges
    type: table
    path: "table:`{domain_catalog_prefix}_payments`.silver.silver_stripe_charges"

Because the dependency is written down, it can be reviewed: before changing a Payments Silver contract, a reviewer can search the repository for the products that link to it. The repository's orchestrator job, pl_data_00_mesh_orchestrator, also uses these declared sources and links to run each system after the systems it reads.

What happens when the data does not match

When a contract runs, LakeLogic applies it to the data. A row that breaks a row rule is written to quarantine with the rule it broke in a _lakelogic_errors column, and the good rows carry on. Dataset rules (uniqueness, null ratios, row counts) are recorded on the run, and stop it if the contract sets fail_pipeline_on_dataset_error. RideFlow's Silver layer sets schema_policy.unknown_fields: drop, so a new column Payments starts sending does not reach consumers until someone adds it to the contract. All of this is enforced at run time, not before deployment.

Go back to the pence example. A new currency column would be dropped at Silver, which keeps the shape stable. Amounts a hundred times too large would pass unless the contract declares a range they break. RideFlow's charge and fare rules check that amounts are not negative and sit below a generous ceiling, which would not catch this. A contract checks what it declares, so a unit change is caught only by a rule written for it, such as a per-currency range or a reconciliation against the ledger.

What is declared, checked and executed

The reference project's claims sit at different levels, and it helps to keep them apart:

• Declared: owners, service expectations, retention and privacy classes live in the contracts and domain files.
• Executed: the Databricks README shows a run of the mesh orchestrator and a quarantine table from the synthetic data. That shows the pipelines ran on that platform; it says nothing about your data or your production load.
• Not established by the files: access isolation. The README says plainly that separate catalogs and declared owners do not set team permissions; Unity Catalog grants must be configured and reviewed. Our post on why a catalog per team is not an access boundary covers that step.

To try the cross-domain read without a cloud account, the RideFlow mesh products notebook (examples/colab/08f_rideflow_mesh_products.ipynb in the LakeLogic repository) builds four domains locally with Polars or DuckDB and joins their published tables. It simulates catalog discovery by scanning folders, so it shows the contracts and joins, not Unity Catalog behaviour.

When you do not need this

• One team, one platform. If the producer and consumer are the same people, platform expectations or dbt tests are usually enough.
• Everything already lives in dbt. Model contracts, access settings and data tests cover much of the interface for dbt-built tables.
• Your problem is access, not content. Start with grants and review; a contract does not restrict who reads a table.
• You need a marketplace and request workflow. A product catalogue such as Data Mesh Manager addresses discovery and approvals, which LakeLogic Core does not provide.

Limits worth knowing

• A contract checks what it declares. Plausible but wrong values pass.
• Schema policy applies when the pipeline runs. It does not stop a producer from changing a source system.
• Declared owners and privacy classes are metadata. Accountability and access control are set elsewhere.
• The RideFlow runs use synthetic data; they show the mechanics, not production performance or business correctness.

Key takeaways

1. Separate three properties. Domain layout, declared ownership and access isolation are established by different things; check each one.
2. Write the interface where both sides review it. The producer's fields and rules and the consumer's links belong in versioned files, not a wiki.
3. Decide what happens at run time. Quarantine failing rows, record dataset results, and choose which failures stop the run.
4. Declare the rule for the failure you fear. A unit change or a duplicate is caught only by a rule written for it.