Databricks MCP server

Databricks is a data lakehouse platform that unifies data engineering, analytics, and machine learning. With this MCP server, AI agents can manage clusters, orchestrate ETL jobs, run SQL queries, manage workspaces, and automate data pipelines through natural language commands.

Setting up an MCP server

This article covers the standard steps for creating an MCP server in AI Gateway and connecting it to an AI client. The steps are the same for every integration — application-specific details (API credentials, OAuth endpoints, and scopes) are covered in the Authentication section on this page.

Before you begin

You'll need:

• Access to AI Gateway with permission to create MCP servers.
• API credentials for the application you're connecting (see the Authentication section on this page for what to collect).

Create an MCP server

Find the app in the catalog

1. Sign in to AI Gateway and select App Catalog from the left navigation.
2. Search for the application you want to connect, then select it from the catalog.
3. Select Create MCP Server to start the wizard.

App Configuration

Confirm the Base URL for the API, then, under Tools, select the endpoints you want to expose. Select Next.

MCP Server Setup

1. Enter a Name for your server — something descriptive that identifies both the application and its purpose.
2. Enter a Description so your team knows what the server is for.
3. Set the log level: choose Production Mode for terser logs, or Non-Production Mode for more verbose logs that can help with debugging.
4. Select Next.

Authentication

Enter the authentication details for the application. This varies by service — see the Authentication section on this page for the specific credentials, OAuth URLs, and scopes to use.

Review

Look over the summary of your MCP server configuration, then select Create & Deploy. AI Gateway provisions the server and provides a server URL you'll use when configuring your AI client.

---

Connect to an AI client

Once your server is deployed, you'll need to add it to the AI client your team uses. Select your client for setup instructions:

• Claude Desktop
• Claude Code
• Cursor
• VS Code
• Windsurf

Tips

• You can create multiple MCP servers for the same application — for example, a read-only server for reporting agents and a read-write server for automation workflows.
• If you're unsure which OAuth scopes to request, start with the minimum read-only set and add write scopes only when needed. Most application pages include scope recommendations.

Authentication

Databricks supports OAuth 2.0 authentication. Configure it in the Databricks Account Console by registering an app connection and obtaining OAuth credentials. Endpoints vary by platform. In the Databricks Account Console, go to Settings > App connections to register an OAuth 2.0 application and obtain your client credentials.

Value	Databricks Cloud (AWS/GCP)	Azure Databricks
Auth endpoint	https://accounts.cloud.databricks.com/oidc/accounts/{account-id}/v1/authorize	https://{workspace-url}/oidc/v1/authorize
Token endpoint	https://accounts.cloud.databricks.com/oidc/accounts/{account-id}/v1/token	https://{workspace-url}/oidc/v1/token
Scopes	all-apis offline_access	user_impersonation offline_access

Replace placeholders with your actual Account ID and workspace URL.

Available tools

The Databricks MCP server exposes cluster management, job orchestration, workspace operations, DBFS access, SQL queries, and user management APIs.

Tool	Purpose
Cluster Management	Create, start, stop, and configure clusters; monitor status and events; resize and scale clusters
Job Orchestration	Create, schedule, and run jobs; monitor runs and get logs; manage job configurations
Workspace Operations	Import and export notebooks; manage folders; organize workspace structure
DBFS Operations	Upload and download files; manage directories; transfer data between systems
SQL Warehouses	Query data with SQL; manage warehouse lifecycle; create dashboards and reports
User & Group Management	Create and update users; manage group membership; configure permissions and roles
Secrets Management	Create secret scopes; store and manage credentials; control access to secrets

Tips

Use auto-scaling to handle variable workloads efficiently and set auto-termination to avoid unnecessary costs.

Choose appropriate instance types for your workload.

Schedule jobs during off-peak hours when possible.

Implement error handling with retries and use conditional logic to skip steps when unnecessary.

Create consistent folder structures for team collaboration and use naming conventions for notebooks and jobs.

Maintain an archive folder for old work.

Batch small files before uploading to improve performance.

Use /FileStore/ for permanent storage.

Regularly clean up temporary files.

Store API keys and credentials in secret scopes rather than notebooks.

Use role-based access controls for secrets and rotate credentials regularly.

---