Making serverless migrations faster, safer, and effortless
by Piyush, Singh, Vijay Raja and Feng Lu
Serverless compute in Databricks offers a simpler, faster way to run your data and AI workloads. It removes the operational burden of managing infrastructure, automatically handling provisioning, scaling, and optimization so teams can focus on their business outcomes. With serverless, organizations can move faster, improve efficiency, and unlock the benefits of cloud-native compute without managing and maintaining complex clusters.
Our serverless usage has grown multifold just in the past year, and today thousands of customers run notebooks and jobs on it. Most teams still have a classic fleet they want to migrate over to the serverless world in order to get the cost and operational benefits without managing infrastructure themselves. The work of moving each workload one at a time often prevents our customers from unlocking serverless benefits for their existing workloads.
You have probably watched this happen. You pick a job, flip it to serverless, and then get hit with an error, an access-mode incompatibility, a Spark config that does not apply, a library…