Databricks Spoke
Summarize
Summary of Databricks Spoke
The Databricks Spoke enables ServiceNow customers to integrate with Databricks directly from their ServiceNow instance. It allows retrieval of details about Databricks warehouses, tables, schemas, and catalogs, as well as execution of SQL statements within Databricks. This integration streamlines automation of data-related tasks by leveraging pre-built actions and sample subflows.
Show less
Key Features
- Databricks Version Support: Built for Databricks Unity Catalog Management API v2.1 and SQL Management API v2.0, compatible with Zurich release and potentially later versions.
- Integration Hub Subscription Required: This spoke requires an active Integration Hub subscription to function.
- Dependencies: Requires installation of dependent ServiceNow plugins such as Complex Object, IntegrationHub Runtime, Action Template - Data Stream, and Action Step - REST. Some may require additional licensing.
- Sample Subflows: Provides example subflows for common tasks including:
- Look up Catalogs
- Look up Schemas within a catalog
- Look up Tables within a schema
- Execute SQL statements and retrieve results
- Automation Actions: Includes actions for executing SQL statements, checking execution status, retrieving result chunks, and listing SQL warehouses, catalogs, schemas, and tables.
- Handling Large Result Sets: To retrieve data over 5MB, customers need to adjust the system property
glide.pf.rest.responsepayloadmaxsizeaccordingly. - Connection and Credential Management: Uses Integration Hub aliases for managing connection and OAuth credentials, simplifying credential updates and supporting multiple environments without reconfiguration of actions.
Practical Benefits for ServiceNow Customers
- Automate data retrieval and SQL execution on Databricks directly from ServiceNow workflows.
- Leverage pre-built subflows and actions to reduce development time and simplify complex integrations.
- Maintain credential and connection consistency across environments with alias management.
- Scale data retrieval efficiently, with configurable limits for large datasets.
Retrieves details of warehouses, tables, schemas, and catalogs, and executes SQL statements in Databricks from your ServiceNow instance.
Request apps on the Store
Visit the ServiceNow Store website to view all the available apps and for information about submitting requests to the store. For cumulative release notes information for all released apps, see the ServiceNow Store version history release notes.
Integration Hub subscription
This spoke requires an Integration Hub subscription. For more information, see Legal schedules - IntegrationHub overview.
Supported versions
This spoke was built for these versions of Databricks, but may be compatible with later versions:
| Action category | Databricks version |
|---|---|
| Unity Catalog Management | API version 2.1 |
| SQL Management | API version 2.0 |
Spoke dependencies
- Complex Object (com.glide.cobject)
- ServiceNow IntegrationHub Runtime (com.glide.hub.integration.runtime)
- ServiceNow IntegrationHub Action Template - Data Stream (com.glide.hub.action_type.datastream)
- ServiceNow IntegrationHub Action Step - REST (com.glide.hub.action_step.rest)
Spoke subflows
The Databricks spoke provides sample subflows to demonstrate automating tasks. To customize a sample subflow, copy it to a new application scope. Available sample subflows include:
| Subflow | Description |
|---|---|
| Look up Catalogs | Retrieves list of all catalogs. |
| Look up Schemas | Retrieves list of all schemas from a specific catalog. |
| Look up Tables | Retrieves list of all tables from a specific schema in a catalog. |
| Sample - Execute Databricks SQL Statement | Executes a SQL statement on Databricks and returns the result columns and data array. |
Spoke actions
The Databricks spoke provides actions to automate tasks when events occurs in your ServiceNow instance. Available actions include:
| Category | Action | Description |
|---|---|---|
| SQL Management | Execute SQL Statement | Executes the provided SQL statement. |
| Look up Execution Status | Retrieves the execution status of a SQL statement. | |
| Look up Result Chunk by Index | Retrieves a specific chunk of the result set for a SQL statement by chunk index. Note: If the data you want to retrieve is more than 5MB, modify the default value of the system property
glide.pf.rest.response_payload_max_size and increase it as per your requirement. |
|
| Look up Warehouses Stream | Retrieves a list of SQL warehouses. | |
| Unity Catalog Management | Look up Catalogs | Retrieves details of the required number of catalogs. Note: To retrieve list of all catalogs, use the Look up Catalogs subflow. |
| Look up Schemas | Retrieves details of the required number of schemas from a specific catalog. Note: To retrieve list of all schemas from a specific catalog, use the Look up Schemas subflow. |
|
| Look up Tables | Retrieves details of the required number of tables from a specific schema in a catalog. Note: To retrieve list of all tables from a specific schema in a catalog, use the Look up Tables subflow. |
Connection and credential alias requirements
Integration Hub uses aliases to manage connection and credential information, and OAuth credentials. Using an alias eliminates the need to configure multiple credentials and connection information profiles when using multiple environments. If the connection or credential information changes, you don't need to update any actions that use the connection.