Databricks Field Guide

Analysing and serving · Chapter 13

Databricks Genie: build a Genie Agent people trust

Reviewed by TechFabric engineers. How we write this guide →

Databricks Genie answers plain-English questions about your data by writing SQL, running it on a SQL warehouse as the person asking, and returning the result with the query behind it. Business users ask through Genie One; the data team builds Genie Agents (called Genie spaces until June 2026), one per subject area, each pointed at a handful of governed tables and taught the business's definitions; and Genie Code is the separate assistant for engineers writing notebooks and SQL.

Ivan Vydrin, a Databricks engineer at TechFabric, built a data-literacy app on Genie that refuses to answer first. In Prove It, a user picks a claim, Genie writes the SQL that would test it, and the app shows that query and makes the user call it before any result appears. Databricks named it a winner in the Creativity & Originality track of its Genie-Powered App Challenge on 10 September 2026 (our write-up).

We think it gets at the reason Genie rollouts stall. A Genie demo answers everything the people who built it ask, about tables they described that morning. A month later the finance director asks about last quarter when the fiscal year starts in July, gets a confident wrong figure, and stops asking. Prove It's answer is to put the evidence on screen before the number, and most of what we'd do on a client's Genie project follows from the same idea.

The short version

Genie answers plain-English questions with SQL that runs against your governed tables, as the person asking, so their row filters and masks apply. You build one Genie Agent per subject area and teach it your definitions, ideally as metric views and verified example queries rather than prose instructions. People use it free until 31 January 2027, but every query bills on the agent's SQL warehouse. Test it with questions its users wrote before an executive sees it, and show people the SQL, because a fluent wrong answer is worse than none.

The names, so you can read what's online #

Most of what's written about Genie predates the June 2026 renames. Genie spaces are now Genie Agents, one per subject area, and they're what your team builds. Genie One is the front door business users open, also in Slack, Teams and the mobile apps. Genie Code is the assistant for people writing notebooks and SQL, billed separately. What was called Research Agent is now agent mode inside a Genie conversation. The API still says "space" in places: the Apps resource key is genie-space and the bundle resource is genie_spaces. What Everything Used to Be Called has the full list.

Whose permissions, and whose warehouse #

A Genie Agent must be attached to a pro or serverless SQL warehouse; classic warehouses aren't supported, and Databricks recommends serverless (set up a Genie Agent). The author needs at least CAN USE on that warehouse, SELECT on the data, and CAN EDIT on the agent. The people asking questions need only SELECT on the tables and CAN VIEW or CAN RUN on the agent, with no warehouse permission of their own.

That split is deliberate, and it's worth understanding before launch. Data access runs with each end user's own Unity Catalog permissions, so the row filters and column masks in Unity Catalog apply on their own: a regional manager sees their region from the same agent a colleague elsewhere uses. Compute follows the author instead. When an author saves the agent's warehouse, that author's compute credentials are embedded, and users query through them. If another author changes the warehouse later, their credentials replace the first. We'd give each agent a warehouse tagged to the team that owns the domain and treat changing it as a cost decision.

Every query Genie writes also runs on a pro or serverless warehouse and bills as an analyst's would. Usage by people is free through 31 January 2027, but service principals are excluded and billed, so the moment an app or a pipeline calls Genie on a person's behalf, that traffic is on the invoice. Four hundred questions a day instead of ten means a busier warehouse, so we'd set its size and auto-stop before the launch email goes out (What It Costs).

decide the rows

pay for the query

Source tables:
federated, later Delta

Unity Catalog views
one definition of each measure

AI/BI dashboards

Genie Agent

Verified example queries
and synonyms

Pro or serverless
SQL warehouse

Asking user's grants,
row filters and masks

Author's embedded
compute credentials

The layout we'd use: Genie and the dashboards read one governed layer, and a Genie query runs with the asker's data permissions on the author's compute

How much does Genie cost? #

Genie bills in two parts, the language model and the warehouse, as of the Genie pricing page on 22 October 2026.

Product Until 31 January 2027 From 1 February 2027
Genie One and Genie Agents, named users Model usage free 150 DBUs per user per month free (about $10.50 in US East), then $0.070 a DBU
Genie Code 150 DBUs per user per month free, then $0.070 a DBU, with 25% off 150 DBUs per user per month free, then $0.070 a DBU
Service principals, any Genie product Billed for all model usage Billed for all model usage
The SQL Genie runs Billed on your warehouse as usual Billed on your warehouse as usual

The free allowance is one pool per user shared between the UI and the API, and it covers model usage only, never the warehouse. The service-principal row is the one that catches app builders, because an app calling Genie with its own identity is billed from its first question. Admins can put budgets and alerts on Genie through Unity Gateway, and a budget never blocks a user's free allowance.

Show the query before the answer #

Prove It is a Streamlit app on Databricks Apps that talks to Genie through the multi-turn Conversation API and reads four Unity Catalog tables through a SQL warehouse. Three details in the public code are worth copying into any app that puts Genie in front of people.

The Genie Agent is declared as an app resource in app.yaml, so the app runs as its own service principal and the agent's ID arrives through the environment. The grant stays visible and reviewable: Can run on the agent, plus USE CATALOG, USE SCHEMA and SELECT on the one schema it reads.

env:
  - name: GENIE_SPACE_ID
    valueFrom: genie-space

A build gate, tests/test_no_sql_in_app_code.py, walks the syntax tree of every application module and fails on any SQL literal, so every query on screen provably came from Genie. And the follow-up ("break that down by department") continues the same Genie conversation, with both queries printed under the same conversation ID, which comes from Genie rather than from the app. In four of the five cases in its docket the verdict changes after that follow-up. The fifth survives on purpose, so the app doesn't teach people that every number is a trick.

A claim the user picks

Genie writes the SQL
that would test it

App shows the query,
holds back the rows

User calls it:
holds up, or a trick

Follow-up in the
same conversation

Genie rewrites its query,
same conversation ID

Verdict settled against
the second result

How Prove It turns a Genie answer into something a user checks before they believe it

The part we'd carry into any client's Genie interface is the order: the generated SQL beside the answer, one click away at most, and the conversation ID kept so a disputed number can be traced back to the query that produced it. Because the app calls Genie as a service principal, its usage is billed under the table above, which is a fair price for a tool whose whole job is to make people check.

What Genie reads before it writes SQL #

Genie's inputs aren't equally reliable, and the order we'd use them in is the opposite of the order most teams reach for them.

Lever When we'd use it How far we'd trust it
Metric view or SQL expression Every measure with a business rule in it: net revenue, margin, active customer Most: Genie reuses the SQL rather than inventing a formula
Verified example query or SQL function The ten questions people ask most, and the multi-step ones High: the answer comes from checked logic
Column comments and synonyms Always, first, because they're cheap High: users say "bookings", the column says order_value
Text instructions Only what SQL can't say, such as "ask which channel before answering about sales" Least, and it gets worse as the list grows

Databricks' own curation guide also calls text instructions a last resort. A model asked for gross margin from raw tables writes a plausible formula that may not be the one finance signed off; Reporting, Semantics, and Power BI makes the wider case for metric views. The guide allows 50 tables, views or metric views per agent and suggests five or fewer; we'd hold to five and pre-join anything wider into views, because every extra table is another join Genie can get wrong.

One set of views under the charts and the questions #

The agent should read the governed views the dashboards read, never the raw tables, so a number from a question and a number on a dashboard come from one definition. Then a change underneath, such as moving from federation to Delta snapshots or a new refresh schedule, never changes what Genie was taught, and the benchmark answers stay valid because the views' meaning doesn't move. Andrey Taran's embedded analytics work on Databricks put five client reports and a natural-language agent over the same two Unity Catalog views, and survived exactly that move; the story is in Real-Time and CDC.

Testing it with the users' questions #

We'd collect thirty to fifty real questions from the people who'll use the agent, each with the SQL that gives the agreed answer, before building anything, because builders tend to write questions the agent can already answer. These are the kinds we'd make sure the set covers.

Kind of question Example What a failure tells you
Business-rule measure "Net revenue after refunds by region" The rule isn't in a metric view or SQL expression yet
Ambiguous time "Sales last quarter", with a July fiscal year A missing fiscal-calendar dimension or clarification rule
Renamed entity A product line under its old name Missing synonyms
Data that isn't there "Margin by supplier" with no supplier table Genie inventing a join; tell it to say it can't answer
Two questions in one "Top customers and their churn risk" A verified query for the multi-step pattern

Genie's built-in benchmarks hold up to 500 questions per agent, and two of their scoring rules catch teams out. An answer with an extra column is rated bad even when every number is right, and above 5,000 rows a result in a different order can be marked bad by truncation, so we'd give benchmark SQL an ORDER BY and keep results small. Detailed results stay visible for a week, so we'd export the scores somewhere that keeps them. We'd also run the set twice, as a broad user and as one behind a row filter, because an answer that's right for the author and wrong for a restricted user is the failure nobody sees in a demo.

Keeping a Genie Agent in version control #

A Genie Agent edited only in the UI has no review and no history, and the benchmark pass rate means little if the agent it measured has since changed. Since Databricks CLI 1.3.0 an agent can be a bundle resource, deployed with databricks bundle deploy like a job. The definition (tables, instructions, example queries) lives in an exported .geniespace.json file next to it, and Genie Agents need the direct deployment engine:

# databricks.yml
bundle:
  name: finance-genie
  engine: direct

variables:
  genie_warehouse_id:
    description: The pro or serverless SQL warehouse the agent runs on

resources:
  genie_spaces:
    finance_revenue:
      title: "Finance: revenue and margin"
      description: "Net revenue, margin and bookings by region and product line"
      warehouse_id: ${var.genie_warehouse_id}
      file_path: ./finance_revenue.geniespace.json
      permissions:
        - level: CAN_RUN
          group_name: finance-analysts

We'd export the agent from the workspace once it passes its benchmarks, commit the JSON, and from then on change it by pull request, with the benchmark run as the check.

Six weeks, as we'd run them #

Week Work Done when
1 One domain, one audience; 30 to 50 real questions with agreed SQL The business owner has signed off the answers
2 Views or metric views for every measure; describe every column; five objects or fewer Every benchmark measure comes from a governed definition
3 Verified queries for the top ten, synonyms, clarification rules; benchmark as two users Every failure has a named cause
4 Fix causes, never the questions; define the agent in a bundle so it's versioned The pass rate holds on a repeat run
5 Pilot with five to ten users; read the Monitoring tab and the "Is this correct?" ratings daily New questions are flowing into the benchmark set
6 Publish the pass rate; agree an owner and a re-run schedule A number people can see, and someone accountable for it

The step teams skip is the last one. A line such as "answers N of our 50 test questions correctly, re-checked every Monday" tells a finance director how far to trust the agent, and it's Prove It's idea at the level of a whole agent: show the evidence next to the answer and let the person decide.

When we wouldn't use it #

When the numbers aren't settled. If finance and sales disagree on revenue, Genie picks one definition and answers confidently with it, so we'd fix the definition in a metric view first. When the question is the same every day, a dashboard answers it faster, cheaper and the same way tomorrow. And when nobody owns the agent, because without someone reading the feedback and adding benchmarks it drifts as the tables change and people quietly stop asking.

For the open-ended question, such as why margin fell in the Midwest, agent mode (formerly Research Agent) builds a plan, runs several queries, adjusts to what they return, and writes a report with citations, charts and the supporting tables; it can also read files in Unity Catalog volumes and call tools through MCP connectors. A faster Flash option is in Beta behind the workspace Previews page. We'd point agent mode at the analysis a person would otherwise spend an afternoon on, and keep standard chat for "what were sales yesterday".