Margin notes · News
Margin notes on Databricks, from the engineers.
What Databricks did, and what we read into it. Funding, pricing, products, renames, and what happens when the platform meets a client's production load. Written by the people doing the work, dated, with a position. RSS.
15 notes
Nothing matched that. Try a broader term, or clear the filter.
- Databricks job completion triggers for Lakeflow Jobs In Beta since 3 November 2026, a Lakeflow Job can start when upstream jobs finish, including other workspaces on the same Unity Catalog metastore. Read
- Databricks Genie Agents: analyze files in Unity Catalog volumes Genie Agents can analyze PDFs and other files in Unity Catalog volumes in Public Preview. What we would attach, and the grant to check first. Read
- Databricks Lakebase Postgres: GA on Google Cloud, and Cloud SQL Lakebase Postgres is generally available on Google Cloud. Five regions, an HA SLA, and what still keeps Cloud SQL in the picture. Read
- Databricks real-time mode for Lakeflow pipelines reaches GA Real-time mode in Lakeflow pipelines is generally available. Use it for a millisecond decision written to Kafka or a ForEach sink, not a Delta table. Read
- Databricks transformWithState async processing: when we'd switch Async processing for Python transformWithState is GA on Databricks Runtime 19. When we would convert a stateful processor, and what to check first. Read
- Databricks serverless jobs from pyproject.toml, and uv.lock beside it Databricks serverless job tasks can now read their environment from pyproject.toml, in Beta. Why we would treat uv.lock as part of the job. Read
- Databricks row filters and column masks: policy metrics now hidden Databricks Runtime 19 hides row filter and column mask internals from Spark Connect metrics. What stays visible, and how we would tune protected tables. Read
- Databricks Lakebase Insights: per-project setup and what stopped Lakebase Insights now runs per project, on a schedule or on demand. Old observability schedules stopped, so what we would turn on and who should see it. Read
- Databricks serverless DataFrame caching: a snapshot you derive from Databricks serverless DataFrame caching is in Public Preview as an eager, session-private snapshot. How it differs from classic caching. Read
- Databricks Genie Code tasks in Lakeflow Jobs: where the agent belongs Genie Code tasks in Lakeflow Jobs are GA. Where we would put an always auto-approved agent in a job graph, and what we would keep out of its reach. Read
- Databricks AI Runtime is GA: training beside governed data AI Runtime, serverless GPUs for training and fine-tuning on Databricks, is generally available. Where we would train on it and the limits to plan for. Read
- Databricks Lakeflow pipeline system tables are GA: cost and refreshes The Lakeflow pipeline system tables are generally available. How we would use them for pipeline spend by name and full refresh audits. Read
- Databricks Feature Views are GA: what changes for feature code Databricks Feature Views reached general availability on 12 October 2026 for batch, streaming and request-time data. What to check before adopting them. Read
- Databricks LTAP Direct Writes in beta: faster synced table loads LTAP Direct Writes loads synced tables straight to storage, off the app's endpoint. What it covers, and the latency test we'd run before trusting it. Read
- Databricks revenue growth: what the $7 billion run-rate means for you Databricks run-rate grew over 80% while customers grew a third. The gap is existing customers spending more. If you are one of them, that growth is your bill. Read