Databricks Field Guide

Start here · Chapter 02

Databricks vs Postgres and a dashboard: when you need more

Reviewed by TechFabric engineers. How we write this guide →

You don't need Databricks if one application database feeds a handful of dashboards: Postgres plus a nightly export and a BI tool is the right architecture at that size. You need a lakehouse when a third source arrives, when two dashboards disagree about the same number, or when someone asks a question that spans systems, because each of those turns the export script into an unowned data platform. Since Lakebase, moving the analytics no longer means moving the application off Postgres.

"We already store data in Firebase, pull it into Postgres and build dashboards on top, and that works fine." It's the objection we'd expect from any team that has shipped a product, and the people raising it are usually right about the thing they're actually saying. This chapter is how we'd decide when that pattern has run out.

The short version

If you have one application, one database and a handful of dashboards, you probably do not need Databricks yet, and anyone who tells you otherwise is selling. The pattern breaks at a predictable point: when the number of data sources goes past a couple, when the dashboards start disagreeing with each other, or when somebody asks a question that spans more than one system. What the pattern really costs is the six months of engineering spent rebuilding it under pressure, far more than any licence, so the useful skill is recognising that point before you hit it.

The pattern being defended #

It goes like this, and it's genuinely reasonable at small scale.

Your application

Firebase or
an app database

Nightly export
script

Postgres

Dashboards
for customers

What most teams build first, and why it works for a while

The application writes into whatever store it uses. Something on a schedule copies that into Postgres, where it's shaped into the tables the dashboards want. A BI tool or a custom React page reads Postgres and shows customers their numbers. The whole thing is maybe four hundred lines of code and one cron job, and it costs almost nothing to run.

For one product, one database and dashboards that a few hundred customers look at, this is very likely the correct answer. It's fast to build, cheap, and every engineer on the team understands all of it. Databricks would be overkill.

Where it actually breaks #

The usual assumption is that you outgrow Postgres when the data gets big, and we'd bet against it: Postgres holds far more than most teams expect. What breaks is everything around it, in the three ways below, and the first one usually arrives before the others.

The export script becomes a data platform nobody owns #

The nightly export starts as a script. Then a source system changes a field, so it grows a special case. Somebody needs yesterday's data corrected, so it grows a backfill mode. A second source appears, so it grows a second export. A run fails silently at 3am and the dashboards show stale numbers for a day, so it grows alerting, and when the alerting is too noisy someone mutes it.

Give it two years and the shape we'd expect is a thousand-line script with no tests, understood by one person, that is the most business-critical code in the company. That cost never shows on a platform comparison because it's paid in engineering time rather than invoices.

The dashboards start disagreeing #

The second break costs trust rather than time.

Once there are two dashboards, there are two definitions. One counts an account as active if it logged in this month, the other if it has an open balance. Both are defensible and neither is written down. A customer sees two numbers that should match and don't, and now every number in the product is suspect.

A better BI tool doesn't fix this. What fixes it is a place where "active account" is defined once, computed once, and read by everything downstream, which is what the medallion architecture is for and what a lakehouse gives you structurally.

Questions that span systems become projects #

The third break is the one that eventually forces the decision.

The moment somebody asks a question that needs the application database, the billing system, the support tool and a spreadsheet finance maintains, the pattern has no answer. Each extra source multiplies the export scripts, and joining across them means loading everything into one Postgres that was never designed to be a warehouse, on hardware you now size for analytics while it also serves your product. We'd treat that request as the signal to decide rather than as another export to write.

What does Databricks replace in a Postgres reporting stack? #

The fair comparison is a product against a kit: on Databricks, one platform covers what the pattern above has you assemble and maintain as six or seven separate pieces.

The same system, built two ways

What you need On Databricks Firebase plus Postgres plus a BI tool What the difference costs you
Getting data in Lakeflow Connect connectors and Auto Loader, declared once A custom export script per source Engineering time, forever, on work nobody owns
Knowing it arrived Pipeline expectations and freshness alerts Whatever you remembered to add Stale dashboards discovered by customers
Defining a metric once A gold table or a metric view, with lineage A SQL view, or the definition inside each dashboard Numbers that disagree
Handling schema change Declared evolution, quarantine, replay The script breaks, or silently drops the field A field that has been wrong for a quarter
Reprocessing history Reprocess from the raw layer The raw data was never kept You cannot fix the past
Serving customer dashboards Gold tables, or a synced table in Lakebase Postgres does both jobs at once Analytics queries slowing the product
Who can see what One Unity Catalog access model Per tool, per database, per dashboard An access review that takes a week
Adding an AI feature The data is already governed and queryable A new export into a vector store A second copy with its own security review

Read the right-hand column rather than the left. Every row is a thing your team will build, and none of them is the product you're trying to ship.

The traffic now runs both ways between Postgres and the lakehouse #

The old framing of this decision was one-directional: analytics outgrows Postgres, so analytics moves. Lakebase, Databricks' managed Postgres, makes the other direction a managed path too, and that changes how a team with an existing Postgres app should sequence the work. Databricks' Lakebase use cases page sets out four patterns, and two matter here.

Lakehouse to Postgres: synced tables. A synced table copies a Unity Catalog table into Lakebase for low-latency reads, on a managed pipeline with no sync scripts. Synced tables are generally available as of October 2026, with three modes:

Mode How it updates Source needs When we'd pick it
Snapshot Full copy every sync Only SELECT * More than about 10% of rows change per cycle, where Databricks puts it at 10x more efficient
Triggered Incremental, on demand or on a schedule A change data feed on the source The default; Databricks calls it expensive below 5-minute intervals
Continuous Streams changes, seconds of lag, 15-second minimum interval A change data feed on the source An application screen that must reflect the lakehouse within seconds

Each sync can hold up to 16 of the database's connections, and they count against the same limit your application uses, so a Postgres serving both needs sizing for both.

Postgres to lakehouse: Lakebase Change Data Feed. In Public Preview, Lakebase Change Data Feed captures row-level inserts, updates and deletes from a Postgres schema into one Unity Catalog Delta table per source table, named lb_<table>_history. It decodes the write-ahead log with a Postgres extension (wal2delta) that runs inside the Lakebase compute, so it isn't free for the database, and flushes changes roughly every 15 seconds. It needs Postgres 16, 17 or 18, at least one row per table, no partitioned tables, and full replica identity on every table in the feed:

-- Lakebase (Postgres): required before a table can join the change data feed
ALTER TABLE public.orders REPLICA IDENTITY FULL;

The history table reads much like Delta's own change data feed, with _pg_change_type (insert, delete, update_preimage, update_postimage), _pg_lsn, _timestamp and a monotonic _sort_by key, so the current state of a table is one query away. The catalog and schema are whatever you chose when you started the feed:

-- Databricks SQL: the latest state of each order, rebuilt from the history table
SELECT * EXCEPT (rn)
FROM (
  SELECT *, ROW_NUMBER() OVER (PARTITION BY order_id ORDER BY _sort_by DESC) AS rn
  FROM analytics_prd.app_history.lb_orders_history
  WHERE _pg_change_type <> 'update_preimage'
)
WHERE rn = 1 AND _pg_change_type <> 'delete';

For the decision in this chapter, that turns "should we move off Postgres?" into "which half of the workload belongs where?". The application backend stays on Postgres with its drivers and frameworks, and the analytical history accumulates in Delta without a Debezium deployment or a nightly dump. The three questions below still decide whether you need a lakehouse at all. They no longer imply migrating the transactional tier. Lakebase covers branching, scale-to-zero and what the Postgres side costs.

What we saw: a copy of production before the conversation ended #

Preetham Reddy, TechFabric's founder, wrote up the moment the transactional side stopped being theory for us. A developer on one of our product teams asked for a copy of production to reproduce a bug; on Lakebase it was a branch, and he had it before the conversation finished. His post, Lakebase vs Snowflake Postgres: branching is the difference, explains why that depends on Lakebase separating compute from storage. For this chapter it means keeping your application on Postgres no longer means keeping it outside the platform.

The three questions that decide it #

We ask these rather than arguing about platforms, because they turn a preference into a decision.

How many sources feed your reporting today, and how many will in a year? One or two is comfortably Postgres territory. Five or more and you're building a data platform whether you call it one or not.

If someone asked you to recompute last quarter's numbers with a corrected business rule, could you? If the answer is no because the raw data was transformed on the way in and the original is gone, you've already lost something a lakehouse would have kept. It's the most useful diagnostic question in this chapter.

When two dashboards disagree, who decides which is right, and where is that decision written down? If the answer is a person rather than a place, the definitions live in people's heads and won't survive them leaving.

When we tell people not to move #

Stay where you are if you have a single application database and reporting that only concerns that application, your data fits comfortably in Postgres, your dashboards are fast, and nobody is asking cross-system questions. Adding a lakehouse to that situation adds moving parts without removing any.

Move when a second or third source has appeared, when someone has been assigned to maintain the pipeline as part of their job, when a number has been wrong in front of a customer, or when the answer to the reprocessing question is no. Those are the signals that the pattern has stopped being simple and has only stayed cheap on the invoice.