Is there a way to connect to azure databricks using JDBC

afraz · ‎08-02-2024

Hi,

i am looking at a way to connect to Azure databricks using JDBC to pull data. Has anyone been able to successfully create a connection with databricks?

afraz · ‎02-18-2025

Okay so i was able to connect to databricks using JDBC. Below are the steps

1. Create a new JDBC data source for Databricks.

2. Create a new Format value for Databricks. Existing ones usually are MySQL/Oracle/SQL Server. Add one more for Databricks

3. In the value field for the new choice value add the below statement

com.databricks.client.jdbc.Driver

4. bring the connection url field on the form of the Data source previously created and add the JDBC connection url directly. you can get it from the databricks administrators. usually looks like below

databricks://xx.azuredatabricks.net:443/default;transportMode=http;ssl=1;httpPath=/sql/1.0/warehouses/xx;AuthMech=11;Auth_Flow=1;OAuth2ClientId=xx;OAuth2Secret=xx;

5. Download the Databricks JAR file from databricks website. the name is DatabricksJDBC42.jar

6. Upload this file to the MID Server from the JAR Files menu item under the MID Server

7. Restart the MID Server

8. Add your SQL statement to the data source previously created and test your data source.

View solution in original post

kmannem · ‎03-20-2025

Hi Vinay Kumar,

Do we need a MID server to establish the connection? We want to pull a column from Databricks and store it in a table that updates automatically.

Do we need any plugins on ServiceNow for this? We are on ITSM standard and have iHub. We also have a MID server for CMDB. Can we use the same MID server, or do we need a different one? Should this MID server have SQL Server Studio on it?

Thanks,

Kushala

Vinay Kumar R · ‎03-24-2025

Please refer to my comments -

Do we need a MID server to establish the connection? We want to pull a column from Databricks and store it in a table that updates automatically.

[Vinay] Please verify the existing Mid-Server access to Databricks. Collaborate with the Databricks team for field mapping and use a transform map to align the fields between staging and target tables.

Do we need any plugins on ServiceNow for this? We are on ITSM standard and have iHub. We also have a MID server for CMDB. Can we use the same MID server, or do we need a different one? Should this MID server have SQL Server Studio on it?

[Vinay] No additional plugins are required. Since this is custom, follow the steps shared in earlier comments. Adjust the Jar file version based on your discussion with the Databricks team.

Sometimes, you may face issues due to incorrect credentials or connectivity (firewall between Mid-Server and Databricks). Collaborate with the Databricks team to obtain the correct credentials and work with the network team to resolve connectivity issues.

Regards,
Vinay

kmannem · ‎04-02-2025

Hi Vinay,

Thank you for the notes. I have set up the data source as mentioned above but ended up receiving the error as mentioned below while testing import. Any suggestions here or Am I missing anything, Any insights would be of great help.

Cannot invoke "java.util.Map.keySet()" because "this.fImportSourceFields" is null

Thanks,

Kushala