Databricks Spoke
Summarize
Summary of Databricks Spoke
The Databricks Spoke enables ServiceNow customers to integrate their instance with Databricks, allowing retrieval of warehouse, table, schema, and catalog details, as well as the execution of SQL statements directly from ServiceNow. This integration streamlines data management and operational workflows by automating interaction with Databricks resources.
Show less
This spoke requires an Integration Hub subscription and supports Databricks versions compatible with Unity Catalog Management API version 2.1 and SQL Management API version 2.0.
Key Features
- SQL Management Actions: Execute SQL statements, check execution status, and retrieve result chunks, supporting large data payloads with configurable system properties.
- Unity Catalog Management Actions: Retrieve detailed information about catalogs, schemas, and tables with options to specify the number of records returned.
- Sample Subflows: Prebuilt subflows demonstrate common automation tasks such as looking up catalogs, schemas, tables, and executing SQL statements. These can be customized by copying into a new application scope.
- Connection and Credential Aliases: Integration Hub uses aliases to manage connection and credential profiles securely across multiple environments, simplifying updates without modifying individual actions.
- Spoke Dependencies: Requires installation of specific ServiceNow plugins including Complex Object, IntegrationHub Runtime, Action Template - Data Stream, and Action Step - REST, some of which may require additional licensing.
Key Outcomes
- ServiceNow customers can efficiently automate data retrieval and SQL execution tasks within Databricks from their ServiceNow instance.
- Improved operational efficiency by leveraging sample subflows and reusable actions to integrate Databricks workflows.
- Secure and manageable connection handling through aliasing, reducing maintenance overhead when managing credentials across environments.
- Flexibility to handle large SQL result sets with configurable payload size limits.
Retrieves details of warehouses, tables, schemas, and catalogs, and executes SQL statements in Databricks from your ServiceNow instance.
Request apps on the Store
Visit the ServiceNow Store website to view all the available apps and for information about submitting requests to the store. For cumulative release notes information for all released apps, see the ServiceNow Store version history release notes.
Integration Hub subscription
This spoke requires an Integration Hub subscription. For more information, see Legal schedules - IntegrationHub overview.
Supported versions
This spoke was built for these versions of Databricks, but may be compatible with later versions:
| Action category | Databricks version |
|---|---|
| Unity Catalog Management | API version 2.1 |
| SQL Management | API version 2.0 |
Spoke dependencies
- Complex Object (com.glide.cobject)
- ServiceNow IntegrationHub Runtime (com.glide.hub.integration.runtime)
- ServiceNow IntegrationHub Action Template - Data Stream (com.glide.hub.action_type.datastream)
- ServiceNow IntegrationHub Action Step - REST (com.glide.hub.action_step.rest)
Spoke subflows
The Databricks spoke provides sample subflows to demonstrate automating tasks. To customize a sample subflow, copy it to a new application scope. Available sample subflows include:
| Subflow | Description |
|---|---|
| Look up Catalogs | Retrieves list of all catalogs. |
| Look up Schemas | Retrieves list of all schemas from a specific catalog. |
| Look up Tables | Retrieves list of all tables from a specific schema in a catalog. |
| Sample - Execute Databricks SQL Statement | Executes a SQL statement on Databricks and returns the result columns and data array. |
Spoke actions
The Databricks spoke provides actions to automate tasks when events occurs in your ServiceNow instance. Available actions include:
| Category | Action | Description |
|---|---|---|
| SQL Management | Execute SQL Statement | Executes the provided SQL statement. |
| Look up Execution Status | Retrieves the execution status of a SQL statement. | |
| Look up Result Chunk by Index | Retrieves a specific chunk of the result set for a SQL statement by chunk index. Note: If the data you want to retrieve is more than 5MB, modify the default value of the system property
glide.pf.rest.response_payload_max_size and increase it as per your requirement. |
|
| Look up Warehouses Stream | Retrieves a list of SQL warehouses. | |
| Unity Catalog Management | Look up Catalogs | Retrieves details of the required number of catalogs. Note: To retrieve list of all catalogs, use the Look up Catalogs subflow. |
| Look up Schemas | Retrieves details of the required number of schemas from a specific catalog. Note: To retrieve list of all schemas from a specific catalog, use the Look up Schemas subflow. |
|
| Look up Tables | Retrieves details of the required number of tables from a specific schema in a catalog. Note: To retrieve list of all tables from a specific schema in a catalog, use the Look up Tables subflow. |
Connection and credential alias requirements
Integration Hub uses aliases to manage connection and credential information, and OAuth credentials. Using an alias eliminates the need to configure multiple credentials and connection information profiles when using multiple environments. If the connection or credential information changes, you don't need to update any actions that use the connection.