The training data for a fine-tuned model usually lives in Unity Catalog, and the GPUs usually live somewhere else, on a Slurm cluster, a Kubernetes pool or a classic GPU cluster somebody sized by hand two years ago. Every run copies governed data out to a stack with its own permissions, its own logs and its own bill. Databricks made AI Runtime generally available on 21 October 2026, serverless on-demand GPUs for training, fine-tuning and other deep learning work inside the workspace, after a Public Preview for single-node tasks that opened in March. That changes where we'd put a training job, and the copy is the first thing we'd want gone.
What we'd move, and why #
The documentation lists four accelerator shapes, each provisioned as a single node. The 1xA10 has 24 GB, the 1xH100 and 8xH100 have 80 GB per GPU, and the 8xB300 has 288 GB per GPU and is still in Public Preview behind a waitlist on AWS. Only the two eight-GPU shapes run distributed training, through the @distributed decorator. We'd send LoRA fine-tuning of a small open model to 1xH100 and keep 8xH100 for the runs that genuinely need the whole node, because eight GPUs waiting on a slow data loader is the most expensive way to sit idle.
What earns the move is governance. Unity Catalog permissions cover the data a workload reads, its checkpoints and its logs, and the workload runs with VM-level isolation, PrivateLink and serverless egress controls. Training data streams from Unity Catalog volumes through UCVolumeDataset, checkpoints go back to a volume, and the run lands in MLflow. On a separate GPU stack, every one of those is a control somebody has to rebuild and keep in step with the catalog.
Spend shows up where the rest of the platform's does. Databricks says AI Runtime usage and costs are tracked through the billing system tables, with workload-level usage records, so we'd wire them into the same cost dashboard as everything else from the first run, not after the first surprising invoice.
The limits we'd plan around #
A CLI job runs for up to 14 days, and connected notebooks and notebook jobs for up to two, so any long run needs checkpointing and a restart plan before it starts. Capacity is on demand and the documentation says it can be constrained in a region; at peak demand AI Runtime may use GPUs in another region, with possible egress costs. It doesn't support the compliance security profile for FedRAMP High or DoD IL5. And a workspace with the compliance security profile needs a workspace admin to turn on the AI Runtime and AI Runtime multi-node H100 previews.
A healthcare scenario #
Take a hypothetical hospital group that wants a small open model to classify discharge summaries, with the notes already governed in Unity Catalog. Before today, the decision was which GPU provider gets a de-identified copy, and who audits that copy. With AI Runtime generally available, we'd keep the notes where they are: fine-tune on 1xH100 reading from a volume, write checkpoints to a volume the same group can read, register the model in Unity Catalog and serve it from Model Serving. The question we'd put to the hospital's security team is whether its workspace runs the compliance security profile, because that decides whether an admin has previews to enable first.
Where we'd start #
Run the quickstart first: a workload.yaml that asks for one A10 and runs nvidia-smi, submitted with databricks air run --file workload.yaml --watch. Then pick one existing GPU job and follow the classic workload migration guide, which swaps TorchDistributor for @distributed and DBFS paths for /Volumes/ paths. Where the model is served afterwards is covered in foundation models, and how its runs are tracked in MLflow.
Questions people ask #
Is Databricks AI Runtime generally available? #
Yes. Databricks announced AI Runtime as generally available on 21 October 2026. The 8xB300 accelerator is still in Public Preview, available on AWS under a waitlist.
What GPUs does Databricks AI Runtime support? #
AI Runtime supports A10, H100 and B300 accelerators, offered as 1xA10, 1xH100, 8xH100 and 8xB300 single-node shapes. Only 8xH100 and 8xB300 support multi-GPU distributed training.
How do I see what AI Runtime costs? #
Databricks says you can track AI Runtime usage and costs through the billing system tables, with workload-level usage records.