Connect your app to Databricks

Connect your app to Databricks to query warehouse data, build live dashboards, and power data-driven tools using your existing tables and views.

Databricks is a data intelligence platform that lets your RShift Sites app work with warehouse data, SQL queries, and cluster resources. The Databricks connector lets you build apps and dashboards on top of your existing Databricks data without exporting CSVs or waiting for engineering tickets.

Use	If you want to
App + chat connector (this page)	Work with a shared Databricks connection in the project chat and in your published apps
App user connector	Let each user of your published app connect their own Databricks account and work with their own data

With Databricks, your app can:

Run SQL queries against your warehouse data
List and manage SQL warehouses
List clusters in your Databricks workspace
Build live dashboards that query data in real time

Authentication uses a Databricks service principal. Your credentials are stored securely in RShift Sites’ gateway and are never exposed to the browser or your app’s frontend code.

Common use cases and example apps

Example app	Example prompt	Description
Live KPI dashboard	Build a dashboard that queries our Databricks warehouse and shows MRR, DAU, and churn rate. Auto-refresh every 5 minutes.	Replace static slides with a live dashboard on your warehouse data. The app queries Databricks directly and displays key metrics that stay up to date without manual exports.
Revenue pipeline tracker	Build a pipeline tracker that pulls revenue and deal data from our Databricks tables and shows a funnel view with filters by region and quarter.	Give RevOps a self-serve view of pipeline data that lives in the warehouse. The app queries Databricks tables where your CRM data lands and presents it in a structured, filterable view.
Team metrics explorer	Build a metrics explorer where users pick a team and date range, then see charts for their key metrics pulled from Databricks.	Let teams explore their own metrics without filing data requests. The app runs parameterized SQL queries and renders results as charts, scoped to each team’s data.
Data quality monitor	Build an internal tool that runs data quality checks against our warehouse tables and flags anomalies.	Catch data issues before they reach downstream consumers. The app runs validation queries on a schedule and surfaces failures in a clean internal view.
Executive summary bot	Build a Slack bot that answers natural language data questions by querying our Databricks warehouse.	Turn your warehouse into a conversational interface for leadership. The app translates questions into SQL, queries Databricks, and posts formatted answers to Slack.

How Databricks connections work

The Databricks connector uses service principal authentication (M2M OAuth). Instead of connecting as an individual user, you create a service principal in Databricks with access to specific tables and views, then provide its credentials to RShift Sites.

What this means for data access

The service principal’s permissions determine what data is available to everyone who uses that connection. RShift Sites does not filter results based on the individual user’s Databricks permissions.

For example, if you create a service principal with access to HR tables, everyone with access to that connection in RShift Sites can query HR data.

Recommended approach: one service principal per access role. Create separate service principals scoped to different data:

databricks-engineering: full warehouse access, only engineers get this connection in RShift Sites
databricks-sales: pipeline and revenue tables only, sales team gets this connection
databricks-company: company-wide safe metrics, everyone gets this connection

RShift Sites controls who can use each connection. Databricks controls what each service principal can query. Together, they provide role-based data access without requiring per-user OAuth.

Databricks uses RShift Sites’ gateway architecture for secure OAuth handling and automatic token refresh. See Gateway-based connectors for details on authentication and usage limits.

How to connect Databricks

Who can create Databricks connections depends on your plan and workspace settings. App + chat connectors are available by default on Free, Pro, and Business plans. On Enterprise plans they are effectively disabled at first: Who can create connections and clients defaults to No one until an admin changes it.

Prerequisites

Before connecting, make sure you have:

A Databricks workspace with at least one SQL warehouse
A service principal configured in Databricks with an OAuth secret (see Databricks M2M OAuth setup)
The service principal’s client ID and client secret
Your Databricks workspace URL (for example, https://dbc-abc123.cloud.databricks.com)
Permission to create connections in your RShift Sites workspace (see Who can create connections and clients)
Network access from RShift Sites to your Databricks workspace, if the workspace restricts inbound traffic

Danger

Databricks IP access lists apply to the workspace REST API, not just the web UI, so an unlisted source IP is rejected before authentication and the connection never verifies. Add RShift Sites’ gateway egress ranges to an ALLOW list, from IP allowlisting. Workspace IP access lists are configured through the Databricks REST API rather than the admin UI, and account-level context-based ingress controls are enforced alongside them, so a request has to pass both.

If the workspace uses front-end Private Link with public network access disabled, the connector cannot reach it at all, and allowlisting is not a workaround: Databricks does not support IP access lists on a workspace with public access disabled. On AWS, public_access_enabled defaults to False in a private access settings object, so confirm this before you start.

Step 1: Configure a service principal in Databricks

If you haven’t already set up a service principal with an OAuth secret, follow these steps in Databricks. For the full reference, see Databricks M2M OAuth setup.

Create a service principal

In the Databricks account console, go to User management → Service principals and create a new service principal. The account console host depends on which cloud hosts your workspace:

Assign it to your workspace

Still in the account console, go to Workspaces, select your target workspace, open its Permissions tab, and click Add permissions. Select the service principal and give it the Workspace access and Databricks SQL access entitlements. You can review and change entitlements later on the service principal’s Configuration tab.

Grant it access to a SQL warehouse

In your Databricks workspace, click SQL Warehouses in the sidebar, open the kebab menu on the row for the warehouse your app will query, click Permissions, then add the service principal with Can use. Can use lets it start the warehouse and run queries.

Generate an OAuth secret

In your Databricks workspace, select your username in the top-right corner and choose Settings.
- Go to Identity and access, then find Service principals and click Manage.
- Select your service principal and open the Secrets tab.
- Click Generate secret, choose an expiry period (maximum 730 days), and click Generate.
- Copy both the Client ID and Client secret immediately. The secret is only shown once. It cannot be retrieved after you close this dialog.

Grant it access to your data

Grant Unity Catalog privileges on the data your app reads. Name the principal with its Application ID (the same value as the OAuth client ID from the previous step), in backticks, not its display name:

    GRANT USE CATALOG ON CATALOG main TO `<application-id>`;
    GRANT USE SCHEMA ON SCHEMA main.analytics TO `<application-id>`;
    GRANT SELECT ON TABLE main.analytics.orders TO `<application-id>`;

All three grants are required. SELECT on a table does nothing without USE CATALOG on its catalog and USE SCHEMA on its schema. Only a catalog owner, or someone with MANAGE on the catalog, can grant USE CATALOG, so you may need to ask them. For the full least-privilege setup, see Restrict a connection to read-only.

Find your Workspace URL

Your Workspace URL appears in the browser address bar when you are signed in to Databricks. Databricks calls its host the instance name, and the format depends on your cloud:

AWS: https://dbc-a1b2345c-d6e7.cloud.databricks.com
Azure: https://adb-5555555555555555.19.azuredatabricks.net. You can also select the workspace resource in the Azure portal and read its URL field.
GCP: https://8757561887652360.0.gcp.databricks.com

See Get identifiers for workspace objects.

Step 2: Set up Databricks connection

Set up the Databricks connection in RShift Sites.

Navigate to Databricks connector

Open Connectors and select Databricks.

Add a new connection

Click Add connection.

Name the connection

In Display name, name the connection (for example, Databricks Engineering or Databricks Sales). Use a name that reflects the access level of the service principal.

Enter your credentials

Use the values from Step 1: Configure a service principal in Databricks:

Workspace URL: your Databricks workspace URL (for example, https://dbc-abc123.cloud.databricks.com)
Client ID: the service principal’s OAuth client ID
Client secret: the service principal’s OAuth client secret

Choose who can use this connection

Under Sharing, the connection is private to you by default and shows a Private label. To share it, click Share with others. Then add workspace members by email, or click Invite entire workspace to make the connection available to everyone in your RShift Sites workspace.

This matters more on Databricks than on most connectors, because the service principal’s grants decide what data is visible. Anyone you add here can query everything the service principal can reach. See Who can use connections and clients for more information.

Connect

Click Connect. RShift Sites verifies the credentials against your Databricks workspace.

When connected, you can link the connection to the projects where you want to use it, and start building apps that query your Databricks data.

Restrict a connection to read-only

The connector forwards whatever your app sends. It applies no method or path allowlist, so any Databricks REST path and any HTTP verb reaches your workspace, including a POST to /api/2.0/sql/statements carrying DROP TABLE. Unity Catalog privileges are the control that works: grant the service principal SELECT and nothing more.

Grant read-only privileges in Unity Catalog

Run these as a catalog owner, or as someone with MANAGE on the catalog. Name the principal with its Application ID in backticks, not its display name:

    GRANT USE CATALOG ON CATALOG main TO `<application-id>`;
    GRANT USE SCHEMA ON SCHEMA main.analytics TO `<application-id>`;
    GRANT SELECT ON SCHEMA main.analytics TO `<application-id>`;

GRANT SELECT ON SCHEMA covers every current and future table and view in the schema. Grant SELECT ON TABLE main.analytics.orders instead when you want to name individual tables. Repeat the statements for each schema your app reads.

Give it a dedicated warehouse

Grant Can use on one warehouse reserved for this connection rather than your shared analytics warehouse. Query volume comes from your app’s end users, so it is unpredictable and outside your control: a dedicated warehouse isolates that load, makes the app’s spend its own line item, and lets you size and time-limit it independently. Set a small Cluster Size and a short Auto stop.

Leave Can manage off. Can use is enough to start the warehouse and run queries, and it does not allow resizing or deleting it.

Keep one service principal per access role

Grants attach to the service principal, so a connection can only ever be as narrow as its principal. If one team needs write access, give that team a second service principal and a second RShift Sites connection instead of widening the read-only one.

Building a semantic layer

Every Databricks use case benefits from a semantic layer: a shared definition of what your key metrics mean, which tables to use, and what assumptions they carry. What counts as a “daily active user”? How is MRR calculated? Which view should be used for churn, and does it exclude trials?

Without this shared context, each app or dashboard risks computing the same metric differently.

If you already have a semantic layer

If your Databricks workspace already has a semantic layer (for example, dbt metrics, Unity Catalog tags, or a YAML definitions file), point RShift Sites to it:

Use our semantic layer at catalog.schema.metrics_definitions when computing any KPIs. MRR is defined there as monthly_recurring_revenue.

If you don’t have one yet

You can build a semantic layer quickly in RShift Sites using a dedicated project. Create a new project, connect it to Databricks, and ask the agent to explore your warehouse and draft definitions:

Let's build a full semantic layer for our Databricks warehouse. Please start by exploring the tables and forming your own understanding of the data. Create a sample dashboard that showcases your analysis of the key metrics, the tables they use, and the assumptions behind each query. We will walk through them together and correct anything that is wrong. Use these tables as the main starting point: [table names]

Drop in any existing context you have (prior dashboards, a data dictionary, a dbt schema file) and RShift Sites will incorporate it. Ask the agent to save the output as Markdown or YAML files in its project directory:

Save the finalized metric definitions as YAML files in this project. Other projects will reference them.

Once saved, other RShift Sites projects in the same RShift Sites workspace can reference that project’s knowledge to get consistent metric definitions out of the box.

Limitations

A service principal connection has no per-user data scoping. Everyone using the connection sees the same data, the service principal’s data. Create separate service principals per access role, or use the Databricks app user connector so each end user queries under their own Databricks login and permissions.
No enforced read-only access. The connector applies no method or path allowlist, so a connection runs whatever SQL its service principal is allowed to run. Scope it with Unity Catalog grants. See Restrict a connection to read-only.
No automatic caching. Query results are not cached by default. You can ask RShift Sites to add caching logic to your app at your chosen interval.
Connection access is not enforced after publishing. Connection-level access decides who can build with the connection, not who can visit the published app. Who can visit is a separate, publish-time control: on Business and Enterprise plans, set the site’s visibility to Workspace or Custom so visitors have to sign in, and set a workspace-wide default in Workspace settings → Privacy & security → Default website access. See website access control.
Customer-managed cost controls. RShift Sites does not impose query cost caps, and neither do Databricks budgets: they scope to your account, workspaces, or resource tags, and they only send email alerts, with up to a 24 hour delay. The controls that actually limit a warehouse are Auto stop, a small Cluster Size, and a STATEMENT_TIMEOUT. Set the timeout for the whole workspace in Settings → Compute → SQL warehouses → SQL Configuration Parameters, since the system default is 172800 seconds (2 days). Per-warehouse timeouts exist but are in Beta and settable only through the SQL warehouses API.

Manage your {connector_0} connection

Connections are managed from Connectors: select {connector_0}, then open the connection.

Unlink projects to remove {connector_0} access from specific projects while keeping the connection available for others. See Unlink projects from a connection for the steps.
Delete the connection to remove it from the workspace entirely. Deleting is permanent. It removes the credentials from all linked projects, and app features that use {connector_0} stop working until a new connection is added. See Delete a connection for the steps and who can delete.

FAQ

Does RShift Sites enforce my Databricks permissions?

No. RShift Sites enforces who on your team can use a connection. The service principal’s access level determines what data is queryable. If the service principal can see HR tables, everyone with access to that connection can query HR tables. Create separate service principals per access role to scope data.

What if someone runs an expensive query?

RShift Sites does not impose query cost caps, so use Databricks-side controls. Start with a small warehouse and scale up as needed, keep Auto stop short (the default is 45 minutes on pro and classic warehouses, 10 minutes on serverless), and set a workspace-level STATEMENT_TIMEOUT so a runaway query is halted well before the 172800 second (2 day) system default. Databricks budgets help you notice overspend but do not stop it: they send email alerts and cannot cap a SQL warehouse.

Is my data cached or stored in RShift Sites?

Not by default. RShift Sites queries Databricks at runtime with no automatic data replication or caching. Caching is opt-in: you can ask RShift Sites to add caching logic to your app at an interval you choose.

What happens when I publish an app that queries Databricks?

The published app queries Databricks through the service principal, so every visitor sees the same data. Connection-level access is not enforced after publishing: it decides who can build with the connection, not who can visit the app. Control the audience at publish time instead. On Business and Enterprise plans you can set the site’s visibility to Workspace or Custom so visitors have to sign in, and admins can set a workspace-wide default. See website access control.

Can someone leak the Databricks credentials?

No. The service principal credentials are stored server-side in RShift Sites’ gateway and are never exposed to the browser or your app’s frontend code.

Troubleshooting

The connection verifies, but every query fails with HTTP 403

This is the most common Databricks setup failure. When you click Connect, RShift Sites verifies the credentials with a single GET 2.0/preview/scim/v2/Me request. That call proves the service principal exists and that its OAuth secret works, and nothing more. It succeeds even when the principal holds no warehouse permission and no table grants, so a connection can report success and then return 403 on its first real query.

Work back through the grants in Step 1:

The principal has no Can use on the warehouse. Grant it under SQL Warehouses → (your warehouse) → kebab menu → Permissions.
The principal is missing Unity Catalog privileges. It needs USE CATALOG, USE SCHEMA, and SELECT, and all three are required. Check what it already holds with SHOW GRANTS `<application-id>` ON CATALOG main;.
The principal was never assigned to the workspace, or lacks the Databricks SQL access entitlement.

The connection fails immediately with an authentication error

Either the OAuth secret expired (the maximum lifetime is 730 days) or the Workspace URL is wrong. Generate a fresh secret in Databricks and reconnect. Make sure the workspace URL is the host you see in the browser when signed in to Databricks, with no trailing path.

The connection cannot reach Databricks at all

RShift Sites’ gateway is blocked at the network layer. See the network prerequisite in Prerequisites: an IP access list or an account-level ingress policy needs RShift Sites’ egress ranges allowlisted, and a workspace with public network access disabled cannot be reached from RShift Sites at all.

Queries time out or a warehouse never wakes up

The service principal needs Can use on the warehouse to start it. Can view and Can monitor are not enough. If long queries are cut off, check the workspace STATEMENT_TIMEOUT in Settings → Compute → SQL warehouses → SQL Configuration Parameters.