Technology1 min read

A builder’s guide to scaling Databricks Apps

By · Published by Everything Blog

In short

System architects can protect application responsiveness by offloading heavy workloads to dedicated background jobs and isolating arbitrary code execution within a Databricks Sandbox. Development teams should calculate capacity needs early by modeling user journeys, such as estimating how concurrent users translate into specific queries per second against a Lakebase backend. Engineers can build these scalable applications in Python or Node.js environments using frameworks like the AppKit TypeScript SDK.

Key points

  • Scaling plans should be established before writing code by defining acceptable user exper…: Scaling plans should be established before writing code by defining acceptable user experiences, expected operational loads, and restart state requirements.
  • Heavy computational workloads and long-running processes must be offloaded to dedicated s…: Heavy computational workloads and long-running processes must be offloaded to dedicated services to keep interface compute responsive.
  • Uncontrolled or user-generated code should be isolated within a Databricks Sandbox to pre…: Uncontrolled or user-generated code should be isolated within a Databricks Sandbox to preserve application reliability.

Sources checked by our analysis pipeline: 1

How to keep your app fast and reliable as usage grows

by Bruno Tafur and Tushar Madan

Congratulations, you've built an app on Databricks Apps, and it's gone from a couple of users to hundreds or thousands across your organization. Your users want it to respond fast and stay up. And your boss may not love having to pay for XLarge compute across a stack of instances.

How do you make an app that runs fast, stays up and handles growing demand? The intuitive answer is more compute, but that only helps once you know where requests spend their time and whether the app is built to scale: its framework, async code and ability to run on several instances, so the platform can easily scale it up or out.

In this guide, we walk through how to scale Databricks Apps across four stages: design; build; test and deploy; and operate (Figure 1).

The focus of this guide is primarily application performance and resiliency at scale; it does not cover deployment automation or performance optimization and scaling of other Databricks services…

Technology