Storing · Chapter 11

LTAP

LTAP stands for lake transactional/analytical processing, and it is the newest structural claim Databricks has made. It deserves a chapter because it is the first genuinely new architectural category in this space for some years, and because the difference between it and the things it resembles is easy to miss.

What Databricks actually announced #

The claim, in their words, is to be the world's first LTAP platform. The substance underneath is worth separating from the naming.

LTAP unifies transactional and analytical workloads on a single copy of data stored in the lake. The distinguishing move is where the unification happens. HTAP tried to unify at the engine layer by making one database serve both, which put the two workloads in competition for the same machine. Zero ETL unifies at the pipeline layer by having the vendor operate the replication, which leaves two copies and a lag. LTAP unifies at the storage layer, which is the only one of the three where the second copy genuinely stops existing.

The components are ones this manual has already covered. Lakebase is the foundation: serverless Postgres running on open object storage, which Databricks says handles around twelve million database launches a day across customers including Block, Superhuman and Zillow. Unity Catalog provides the governance across both sides. The table formats are Delta and Iceberg, which is what makes one copy readable by two different kinds of engine in the first place.

Alongside the announcement, Databricks named cross-cloud disaster recovery, git-style branching, snapshots, and autonomous database operations as capabilities of the same layer. Branching is covered in Lakebase, and it is the one we would test first.

Why the agent argument is the real argument #

The press release frames LTAP around AI agents rather than around reporting, and that framing is correct rather than opportunistic.

A dashboard tolerates staleness. A person looking at a quarterly trend does not care whether the data is four hours old, and a nightly pipeline is an entirely reasonable way to serve them. This is why the warehouse era lasted as long as it did: for its actual users, the lag was acceptable.

An agent does not tolerate staleness in the same way, because an agent reads and then acts. If a system reads a loan's balance, decides a payoff quote is correct, and issues it, the gap between the read and the truth is not a reporting inconvenience, it is a wrong action taken in the world. The tighter the loop between reading and acting, the less a replication lag is something you can reason about and the more it is a correctness bug.

That is why we regard LTAP as significant rather than as a rename. It is not a faster pipeline. It removes the window in which the two copies can disagree, and that window is exactly where agentic systems fail.

How it differs from the alternatives, precisely #

The honest concession is that Zero ETL is good, and for a great many workloads the difference between two copies eight seconds apart and one copy is not worth changing platforms over. Where it starts to matter is when the number of source systems grows, because Zero ETL is a per-pair arrangement: every source you add is another managed replication to configure, monitor and pay for, and the governance model still lives in whichever service you are reading from at that moment.

What it looks like in practice #

Three worked examples, from the simplest to the one that actually changes an architecture. Each is a shape we have either built or would build, rather than a hypothetical.

One: the order status page nobody has to synchronise #

An application needs to show a customer the current state of their loan, which is a point read, and the servicing team needs to report on loan states by stage, which is a scan. In the warehouse era those are two systems and a pipeline.

The application writes to Lakebase, using ordinary Postgres.

CREATE TABLE loan_state (
  loan_id    TEXT PRIMARY KEY,
  dealer_id  TEXT          NOT NULL,
  stage      TEXT          NOT NULL,
  principal  NUMERIC(12,2) NOT NULL,
  updated_at TIMESTAMPTZ   NOT NULL DEFAULT now()
);

CREATE INDEX loan_state_dealer_idx ON loan_state (dealer_id, stage);

The application reads it the same way, in single-digit milliseconds.

SELECT loan_id, stage, principal, updated_at
FROM loan_state
WHERE loan_id = $1;

The analytical side reads the same data through the catalog, with no export job in between and nothing to be stale.

SELECT
  stage,
  count(*)        AS loans,
  sum(principal)  AS principal_outstanding,
  max(updated_at) AS most_recent_change
FROM originations_prd.operational.loan_state
GROUP BY stage
ORDER BY principal_outstanding DESC;

The thing worth noticing is what is absent. There is no CDC connector, no staging table, no watermark column, no nightly job, no alert for when that job fails, and no conversation about how stale the dashboard is allowed to be.

Two: the agent that reads and then acts #

This is the case the announcement is really about, and the one where a replication lag stops being cosmetic.

An agent handles a payoff quote request. It has to read the current balance, decide whether a quote can be issued, and then issue it. With two copies eight seconds apart, the balance it reads may already be wrong by the time it acts, and the customer receives a quote for a loan that was paid down in between.

The read and the write land on the same copy, so there is no window in which the agent's view and the system of record disagree. The proposal still passes through policy evaluation before anything is issued, because removing the staleness problem does not remove the governance problem, and Governed Actions is still the chapter that applies.

Three: the feature computed in batch and read at request time #

A risk model needs features that are expensive to compute, such as a rolling ninety day payment behaviour summary, and it needs them in a few milliseconds at scoring time.

The batch side computes them into a gold table on whatever cadence makes sense.

CREATE OR REPLACE TABLE originations_prd.gold.dealer_risk_features AS
SELECT
  dealer_id,
  avg(days_late)                                  AS avg_days_late_90d,
  sum(CASE WHEN days_late > 30 THEN 1 ELSE 0 END) AS late_over_30_90d,
  count(*)                                        AS payments_90d,
  current_timestamp()                             AS computed_at
FROM originations_prd.silver.payment
WHERE payment_date >= current_date() - INTERVAL 90 DAYS
GROUP BY dealer_id;

The serving side reads one row of it per request, at the latency an API needs. The direction of dependency stays one way: the lakehouse computes, the operational side serves, and nobody writes features back the other way. Serving Data to Applications covers that discipline in full.

A worked comparison of the same requirement #

It is easier to see the difference in a requirement than in an architecture diagram. Take the first example: an application that writes loan state, and a dashboard that reports on it, correct to the minute.

The last row is the one experienced engineers react to. Replication is the part of that architecture that fails most often and is understood by the fewest people, and every one of the three alternatives has it while the first does not.

What it does not remove #

Three things people assume LTAP solves and it does not, worth stating so nobody plans around them.

It does not remove the need for a medallion structure. Raw operational tables are still shaped for the application, not for analysis, and the conformance and definition work in The Medallion Architecture still has to happen. LTAP removes the copy, not the modelling.

It does not remove ingestion from systems that are not on the platform. Your Salesforce data, your ERP, your partner's file drop and your Cosmos DB container are all still outside, and Real-Time and Change Data Capture still applies to them. LTAP unifies the data you keep in Lakebase and the lakehouse, not the data somebody else owns.

It does not make an analytical query fast on a table laid out for transactions, or the reverse. Physical layout still matters, and Tables and Storage still applies.

What we would do about it now #

The capability is described as coming soon as part of Lakebase, while the components underneath already run production workloads. That shapes the sensible response.

We would not restructure a working platform in anticipation of it. We would, on any new build where an application and its analytics are being designed together, keep the option open by putting the operational store in Lakebase rather than in a separate managed Postgres, since that is the decision that is expensive to reverse later and cheap to make now.

And we would test the claim rather than accept it. Put a real transactional workload and a real analytical workload on the same data, then measure the transactional latency while the analytical scan runs. That number is the whole difference between LTAP and HTAP, and it is measurable in an afternoon.

Table of Contents