Reference · Chapter 32
Databricks glossary: terms, old product names and meanings
Reviewed by TechFabric engineers. How we write this guide →
This Databricks glossary defines the terms a developer meets on the platform, from account and workspace through Unity Catalog, Lakeflow and Lakebase to Genie and Model Serving, each in one sentence with the current product name as of October 2026. Where Databricks renamed a product, the old name is given too, because the old name is usually what's in your code, your bill and the tutorial you found. Each entry links to the chapter that covers it in depth.
Renames has the full table of old and new names with where each old one survives. For the ideas the rest of the guide rests on (OLTP and OLAP, the lakehouse, HTAP, Zero ETL and LTAP), the LTAP chapter explains them properly and this page only points there.
Terms teams mix up #
These are the pairs we'd untangle first with a team new to the platform, because each confusion leads to a real mistake and not just a wrong word.
| People say | When they mean | The difference that matters |
|---|---|---|
| Workspace | Account | The account owns identity, the metastore and billing; a workspace is where people and compute live. Isolate environments and regions with workspaces, never teams. |
| Schema | Database | The same object. Databricks called it a database and now calls it a schema; SQL accepts both keywords. |
| Managed table | External table | Databricks owns a managed table's files and lifecycle, so dropping it deletes the data and predictive optimization can maintain it. We'd default to managed and keep external tables for files another system must also write. |
| Streaming table | Materialized view | A streaming table processes each input record once and suits ingestion; a materialized view recomputes a result and suits aggregates. The pipelines API makes you pick with @dp.table or @dp.materialized_view. |
| Job | Pipeline | A pipeline declares tables and their dependencies; a job runs tasks on a schedule or trigger, and one of those tasks is often a pipeline. You usually need both. |
| Delta Sharing | Lakehouse Federation | Sharing gives someone else read access to your tables. Federation lets you query someone else's database in place. Opposite directions. |
| Row filter | A filtered view | A row filter is attached to the table, so every reader gets it; a view protects only the people who query the view. We'd use row filters for anything that must hold for every consumer. |
| AI Search | Genie | AI Search finds passages similar to a question for an application to use. Genie answers business questions over tables by writing SQL. |
| Unity Gateway | AI Gateway | AI Gateway (legacy) governs one serving endpoint. Unity Gateway governs model and MCP traffic for the whole account from one control plane. |
Naming we'd standardise on day one #
A glossary for the platform is half the job; the other half is the names your own estate uses, and those are cheap to fix before the first pipeline and expensive after the fiftieth. This is the convention we'd propose to a new team, and each line removes an argument we'd otherwise expect to have later.
-- Catalog = domain plus environment, so a grant can never cross environments by accident
CREATE CATALOG IF NOT EXISTS sales_dev;
CREATE CATALOG IF NOT EXISTS sales_prd;
-- Schema = medallion layer, so the promise a table makes is in its name
CREATE SCHEMA IF NOT EXISTS sales_prd.bronze;
CREATE SCHEMA IF NOT EXISTS sales_prd.silver;
CREATE SCHEMA IF NOT EXISTS sales_prd.gold;
-- Groups own things and get grants; people never do
GRANT USE CATALOG, USE SCHEMA, SELECT ON CATALOG sales_prd TO `sales-readers`;
ALTER SCHEMA sales_prd.gold OWNER TO `sales-data-owners`;Three rules sit behind it. Environment goes in the catalog name, never in a table name, because a table called orders_dev sitting in a production catalog is how test data reaches a board report. A gold table is named for its consumer (gold.finance_monthly_revenue, not gold.revenue_v2), so retiring it is a conversation with one team. And production objects belong to a group, never to the engineer who created them, because ownership that leaves with a person takes access with it.
The machinery #
Notebook: A document of runnable cells that mixes code, results and prose, with versions and permissions. We'd keep transformation logic in importable files and use the notebook as the thin thing that calls them, because that's what makes the logic testable (CI/CD and Environments).
Apache Spark: The open source distributed engine under Databricks' analytical side, created by the founders of Databricks. It spreads one query's work across many machines. On serverless compute you no longer tune it, and we'd count that as a feature: most hand-set Spark configuration in an old estate was set once to fix one job and never revisited.
Cluster: A group of machines running Spark for you, one driver coordinating many workers. An all-purpose cluster is for people; a job cluster exists for one run. A scheduled job on an all-purpose cluster is the most common avoidable cost we'd look for (Cost and Performance).
SQL warehouse: Compute packaged for SQL queries, sized in t-shirt sizes from 2X-Small up, in serverless or classic form. See Reporting, Semantics, and Power BI.
Serverless compute: Compute that Databricks keeps warm and allocates in seconds, so there's nothing to size, start or switch off. Our default for new work.
DBU: The Databricks Unit, the billing unit, consumed per second while compute runs at a rate set by the compute type. See What It Actually Costs.
Photon: Databricks' vectorised query engine, written in C++. On classic compute it bills at a higher DBU rate and is usually cheaper overall, because the work finishes sooner.
Parquet: The columnar file format that Delta and Iceberg tables keep their data in. Never read a Delta table's Parquet files directly: without the transaction log you get files that were logically deleted or replaced, and the numbers look plausible.
System tables: Read-only tables under the system catalog that record billing, audit, lineage, job runs and query history. system.billing.usage still labels pipeline spend DLT; Renames has the query that relabels it.
Platform and governance #
Account: The top-level Databricks entity that owns identity, metastores and billing. See Unity Catalog.
Workspace: The environment where people, notebooks, jobs and compute live. An isolation boundary for environments and regions, not for teams.
Metastore: The account-level container for catalogs, one per region. Because a workspace can only attach to the metastore in its region, the region you pick for a workspace is a governance decision as much as a latency one.
Unity Catalog: Databricks' governance layer for data and AI assets, covering access control, lineage and audit across every workspace in a region. See Unity Catalog.
Catalog: The top of the three-part catalog.schema.object namespace. We use it as the domain and environment boundary, so sales_prd and sales_dev are separate catalogs.
Schema: The middle of the namespace, previously called a database. We use it as the medallion layer or the data product boundary.
External location: A registered storage path plus the credential that may read it, which is how Unity Catalog governs files it doesn't manage.
Row filter and column mask: SQL functions attached to a table or column that remove rows or transform values depending on who's asking, evaluated at query time. See Unity Catalog and Multi-Tenant and Per-User Reporting.
ABAC: Attribute-based access control, where one policy applies row filters or column masks to every table carrying a governed tag, instead of being attached table by table. Some parts are still in Beta as of October 2026, including metastore-level and DENY policies. See Security.
Service principal: A machine identity. Production jobs and deployments should run as one, so the deployment path doesn't leave with an engineer.
Data and storage #
Delta Lake: The default table format on Databricks: Parquet files plus a transaction log, giving atomic writes, time travel and concurrent readers. See Tables and Storage.
Apache Iceberg: The other open table format. Unity Catalog manages Iceberg tables natively, serves them to outside engines through its Iceberg REST catalog, and reads Iceberg tables owned by other catalogs (Glue, Hive, Snowflake Horizon) as read-only foreign tables. See Tables and Storage.
Managed table and external table: See the table at the top of this page.
Liquid clustering: The data layout that replaces Hive-style partitioning and ZORDER for most tables, with clustering keys you can change without rewriting the table. We'd reach for it before partitioning on anything new.
Predictive optimization: Compaction, statistics and clustering maintenance that Databricks runs on managed tables, driven by how they're queried instead of by a schedule you own.
Time travel: Reading or restoring a Delta table as of an earlier version or timestamp, within its retention settings. A real safety net for one table, and no help for the tables computed from it.
Deletion vector: A marker recording that a row is deleted, written without rewriting the file that holds it, which makes deletes and updates far cheaper.
Shallow clone: A copy of a table that shares its source's files until written to. How we'd give every branch a production-shaped test table.
Lakebase: Managed Postgres on Databricks, with storage separated from compute, autoscaling that suspends idle compute, and the same governance boundary as the lakehouse. See Lakebase.
Branch (Lakebase): A copy of a Lakebase database made by pointer on copy-on-write storage, so it's ready in seconds and costs little until something diverges. We'd give every pull request that touches a schema its own branch to migrate against.
Synced table: A Unity Catalog table kept in step with a Lakebase Postgres table, so applications read from Postgres at low latency while analytics reads the Delta side.
Pipelines and orchestration #
Auto Loader: Incremental file ingestion from object storage that tracks which files it has processed and handles schema inference and evolution. See Real-Time and Change Data Capture.
Lakeflow Connect: Databricks' managed connectors that pull from SaaS applications and databases without ingestion code of your own.
Ingestion gateway: The Lakeflow Connect component that reads snapshots, change logs and metadata from a source database.
Lakeflow pipelines: Databricks' declarative pipeline framework, formerly Delta Live Tables (DLT), built on Apache Spark Declarative Pipelines; the docs title it "Spark Declarative Pipelines on Lakeflow". Python code imports from pyspark import pipelines as dp, and import dlt still works. See Medallion Architecture and Renames.
Apache Spark Declarative Pipelines: The open source pipeline API in Apache Spark from 4.1, which Lakeflow pipelines extend. Tables, materialized views and views are portable; expectations and AUTO CDC are Databricks only.
Expectation: A data quality rule declared on a pipeline table, enforced by recording a warning, dropping the row or failing the update. We'd quarantine instead of dropping, so the losses stay visible.
AUTO CDC: The declarative way to apply change records to a target table, with keys, a sequencing column and delete handling, for SCD type 1 and type 2. Formerly APPLY CHANGES; in Python, dp.create_auto_cdc_flow. See Real-Time and Change Data Capture.
Change data capture: Reading a source's change log so a target stays in step without re-reading everything. See Real-Time and Change Data Capture.
Change feed: A source system's own ordered record of changes, such as the Azure Cosmos DB change feed, or a Delta table's change data feed.
Lakeflow Jobs: Databricks' orchestrator, formerly Databricks Workflows: multi-task graphs that run on a schedule, a file arrival or a table update. See Orchestration.
Declarative Automation Bundles: Formerly Databricks Asset Bundles. Jobs, pipelines and other resources defined in databricks.yml with per-environment targets, deployed with databricks bundle deploy. See CI/CD and Environments.
Lakehouse Federation: Querying an outside database in place through Unity Catalog, without ingesting it, either by pushing queries down over JDBC or, for some sources such as Snowflake, by reading the table files directly. We'd use it to get a first dashboard on real data within days, then move anything queried all day into Delta, because every pushed-down query is load on the source. See Migration.
Anomaly detection: Data quality monitoring, in Public Preview as of October 2026, that learns each table's commit pattern and flags tables that are stale or incomplete, switched on per schema. See Observability.
Medallion layers #
Medallion architecture: Organising tables into bronze, silver and gold layers, where each layer makes a stronger promise about the data. See The Medallion Architecture.
Bronze: Raw, as received, append-only, with ingestion metadata. It promises fidelity.
Silver: Typed, deduplicated, validated and conformed. It promises correctness.
Gold: Shaped for a named consumer. It promises fitness for that consumer.
Quarantine: The table where records that fail validation go, with the reason attached. The layer names are worth little without these promises; The Medallion Architecture covers enforcing them.
Analytics and reporting #
Metric view: A business measure and its dimensions defined once (YAML inside a CREATE VIEW ... WITH METRICS statement) and registered in Unity Catalog, so a dashboard, an analyst and an agent all read one definition. Metric views are the core of Unity Catalog semantics, with agent metadata (synonyms, display names, formats) beside them; the docs moved to /uc-semantics/metric-views/. See Reporting.
AI/BI dashboards: Databricks' built-in dashboards, formerly Lakeview dashboards, which run on governed tables and metric views with no extract; the REST API still says lakeview.
Genie: Databricks' conversational family. Genie Agents (formerly Genie spaces) answer questions over the datasets an analyst curates for one subject, Genie One is the single chat for business users across data, dashboards and apps, and Genie Code is the assistant for developers. See Genie.
Knowledge store: A Genie Agent's curated context: table and column descriptions, synonyms, join relationships, SQL expressions and prompt matching. It doesn't change Unity Catalog metadata, and in our view the quality of this curation decides whether people trust the answers.
Databricks Apps: Python or Node.js web applications hosted inside a workspace, running as the app's own service principal or on behalf of the signed-in user against governed data. See Databricks Apps.
Delta Sharing: Read access to your tables for another organisation without copying the data, where the recipient doesn't need to be on Databricks. Open sharing authenticates either with a bearer token or credential file, which is only as safe as the recipient's handling of it, or with OIDC federation from the recipient's identity provider. See Sharing.
Clean room: A governed space where two organisations compute over combined data without either seeing the other's rows.
Models and generative AI #
MLflow: Experiment tracking, model and prompt versioning, tracing and evaluation, open source and managed on Databricks. See MLflow.
Model alias: A moving pointer to a model version, such as champion, which is how a model is promoted without copying it.
Model Serving: Managed REST endpoints for foundation models and your own models.
Inference table: A governed table of a serving endpoint's requests and responses, which turns production traffic into evaluation data.
Foundation Model APIs: Databricks-hosted models, billed pay-per-token or by provisioned throughput. See Foundation Models.
Pay-per-token and provisioned throughput: Pay for tokens on a shared endpoint, or reserve capacity for predictable latency. We'd start on pay-per-token and move only once real traffic justifies the reservation.
Unity Gateway: Formerly Unity AI Gateway, the account-level control plane for AI traffic that routes model and MCP requests and applies rate limits, guardrails, usage tracking and cost controls. The per-endpoint gateway it replaces is labelled AI Gateway (legacy). See Foundation Models.
AI Functions: SQL functions that call a model, such as ai_query and ai_classify, so classification or extraction runs inside a query or pipeline.
ai_decide: An AI Function in Beta, announced on 30 September 2026, that answers structured questions about each row (a choice, a probability or a score) with a decision model.
Decision model: A model that returns decisions and their probabilities instead of text. Faster and cheaper than a language model for classification, scoring and routing.
AI Search: Databricks AI Search, formerly Vector Search and Mosaic AI Vector Search: a managed index over Unity Catalog tables that stays in sync with its source. The current Python client is AISearchClient. See Context and Retrieval.
Agent Bricks: Databricks' tooling for building and evaluating agents. Its Knowledge Assistant and Supervisor Agent are now legacy; new low-code agents are Genie Agents, and code-based ones start from the Agent Bricks CLI. See Agents.
Chunk: A passage indexed as one retrievable unit. We'd chunk on the document's own boundaries, such as headings and clauses, not on a character count.
Reranking: Narrowing a wide set of retrieved candidates to the few that go into the prompt.
Context window: The tokens a model can attend to in one call, shared between instructions, tools, retrieved text, history and the answer. A budget to plan, not a container to fill.
Groundedness: Whether an answer is supported by the context actually retrieved, which is a different question from whether it happens to be true.
Terms this guide introduces #
Governed action: A proposed change to a system of record that is evaluated against policy before it runs, returning pass, warn, review or block with evidence. Warn proceeds and is recorded; review waits for a named person. See Governed Actions.
Decision evidence: The record a policy evaluation leaves behind: the decision id, the policy revision and a hash of the inputs.
Fabric Harness: TechFabric's open source, Apache-2.0 TypeScript framework for agents that act on governed data, built on durable execution.
What's popular