Start here · Chapter 03
Why Not Just Postgres and a Dashboard?
This is the most common objection we hear, and it deserves a serious answer rather than a sales one, because the people raising it are usually right about the thing they are actually saying.
The short version
If you have one application, one database, and a handful of dashboards, you probably do not need Databricks yet, and anyone who tells you otherwise is selling. The pattern breaks at a specific and predictable point: when the number of data sources goes past a couple, when the dashboards start disagreeing with each other, or when somebody asks a question that spans more than one system. This chapter is about recognising that point before you hit it, because the cost of the pattern is not the software licence, it is the six months of engineering you spend rebuilding it under pressure.
The pattern being defended #
It goes like this, and it is genuinely reasonable at small scale.
Data is written by the application into whatever store the application uses. Something on a schedule copies it into Postgres, where it is shaped into the tables the dashboards want. A BI tool or a custom React page reads Postgres and shows customers their numbers. The whole thing is maybe four hundred lines of code and one cron job, and it costs almost nothing to run.
We want to be clear that this is a good architecture for what it is. It is fast to build, easy to reason about, cheap, and every engineer on the team understands all of it. If your situation is one product, one database, and dashboards that a few hundred customers look at, this is very likely the correct answer and Databricks would be overkill.
Where it actually breaks #
The failure is not gradual and it is not about volume. Teams assume they will outgrow Postgres when the data gets big, and that is almost never what happens. Postgres will hold far more data than most people expect. What breaks is everything around it.
The export script becomes a data platform nobody owns #
The nightly export starts as a script. Then a source system changes a field, so it grows a special case. Then somebody needs yesterday's data corrected, so it grows a backfill mode. Then a second source appears, so it grows a second export. Then a run fails silently at 3am and the dashboards show stale numbers for a day, so it grows alerting. Then the alerting is too noisy, so someone mutes it.
Two years in, that script is a thousand lines, has no tests, is understood by one person, and is the single most business-critical piece of code in the company. This is the actual cost of the pattern, and it is invisible on any cost comparison because it is paid in engineering time rather than in invoices.
The dashboards start disagreeing #
The second break is more damaging because it costs trust rather than time.
Once there are two dashboards, there are two definitions. One counts an account as active if it logged in this month, the other if it has an open balance. Both are defensible. Neither is written down. A customer sees two numbers that should match and do not, and now every number in the product is suspect.
The fix for this is not a better BI tool. It is a place where "active account" is defined once, computed once, and read by everything downstream, which is what the medallion architecture is for and what a lakehouse gives you structurally.
Questions that span systems become projects #
The third break is the one that eventually forces the decision.
The moment somebody asks a question that requires the application database, the billing system, the support tool, and a spreadsheet finance maintains, the pattern has no answer. Each additional source multiplies the export scripts, and joining across them means loading them all into one Postgres that was never designed to be a warehouse, on hardware you now have to size for analytics while it also serves your dashboards.
What Databricks actually replaces #
The honest framing is not that Databricks is better than Postgres. It is that Databricks is one product where the pattern above is six or seven things you assemble and maintain yourself.
The same system, built two ways
| What you need | On Databricks | Firebase plus Postgres plus a BI tool | What the difference costs you |
|---|---|---|---|
| Getting data in | Managed connectors and Auto Loader, declared once | A custom export script per source | Engineering time, forever, on work nobody owns |
| Knowing it arrived | Freshness and volume expectations, alerted | Whatever you remembered to add | Stale dashboards discovered by customers |
| Defining a metric once | A gold table with lineage | A SQL view, or the definition inside each dashboard | Numbers that disagree |
| Handling schema change | Declared evolution, quarantine, replay | The script breaks, or silently drops the field | A field that has been wrong for a quarter |
| Reprocessing history | Reprocess from the raw layer | The raw data was never kept | You cannot fix the past |
| Serving customer dashboards | Gold tables into a serving layer | Postgres does both jobs at once | Analytics queries slowing the product |
| Who can see what | One access model over all of it | Per tool, per database, per dashboard | An access review that takes a week |
| Adding an AI feature | The data is already governed and queryable | A new export into a vector store | A second copy with its own security review |
Look at that right-hand column rather than the left. Every row is a thing your team will build, and none of them are the product you are actually trying to ship.
The three questions that decide it #
We ask these rather than arguing about platforms, because they turn a preference into a decision.
How many sources feed your reporting today, and how many will in a year? One or two is comfortably Postgres territory. Five or more and you are building a data platform whether you call it one or not.
If someone asked you to recompute last quarter's numbers with a corrected business rule, could you? If the answer is no because the raw data was transformed on the way in and the original is gone, you have already lost something a lakehouse would have kept. This is the single most useful diagnostic question in this chapter.
When two dashboards disagree, who decides which is right, and where is that decision written down? If the answer is a person rather than a place, the definitions live in people's heads and will not survive them leaving.
When we tell people not to move #
We would rather say this plainly than have you discover we were overselling.
Stay where you are if you have a single application database and reporting that only concerns that application, if your data fits comfortably in Postgres and your dashboards are fast, and if nobody is asking cross-system questions. Adding a lakehouse to that situation adds moving parts without removing any.
Move when a second or third source has appeared, when someone has been assigned to maintain the pipeline as a real part of their job, when a number has been wrong in front of a customer, or when the answer to the reprocessing question above is no. Those are the signals that the pattern has stopped being simple and has only stayed cheap on the invoice.