JDBC Connection Issue - Did not get a response from the MID Server

Heiko Bllr — Wed, 23 Oct 2019 09:45:06 GMT

Hi,

we have a several JDBC datasources configured, for SQL Server and Oracle databases.

From time to time it seems the MID Server is not able to establish a connection to the database after retrying for 5 mins (every 5s).

The errors we see in the agent.log would look like this:

10/22/19 19:20:10 (189) Worker-Standard:JDBCProbeError-a807251fdbec0094497151c6f49619a0 SEVERE *** ERROR *** java.lang.NullPointerException

at com.service_now.mid.probe.JDBCProbe.removeCachedConnection(JDBCProbe.java:271)

at com.service_now.mid.probe.JDBCProbe.cancel(JDBCProbe.java:806)

at com.service_now.mid.message_executors.CancelProbeExecutor.cancelProbe(CancelProbeExecutor.java:58)

at com.service_now.mid.probe.JDBCProbeError.probe(JDBCProbeError.java:35)

at com.service_now.mid.probe.AProbe.process(AProbe.java:96)

at com.service_now.mid.queue_worker.AWorker.runWorker(AWorker.java:121)

at com.service_now.mid.queue_worker.AWorkerThread.run(AWorkerThread.java:20)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

...

10/22/19 19:20:10 (194) Worker-Standard:JDBCProbeError-a807251fdbec0094497151c6f49619a0 Worker completed: JDBCProbeError source: Did not get a response from the MID server time: 0:00:00.002

...

10/22/19 19:20:15 (174) Worker-Standard:JDBCProbeError-8507e91fdbec0094497151c6f4961965 Worker starting: JDBCProbeError source: Did not get a response from the MID server

10/22/19 19:20:15 (176) Worker-Standard:JDBCProbeError-8507e91fdbec0094497151c6f4961965 Cancellation called on probe com.service_now.mid.probe.JDBCProbe

10/22/19 19:20:15 (176) Worker-Standard:JDBCProbe-aed5e5dbdbec0094497151c6f49619e5 SEVERE *** ERROR *** Error when trying to execute: SELECT * FROM [removed] 10/22/19 19:20:15 (177) Worker-Standard:JDBCProbe-aed5e5dbdbec0094497151c6f49619e5 SEVERE *** ERROR *** Socket closed

com.microsoft.sqlserver.jdbc.SQLServerException: Socket closed

at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1667)

at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1654)

at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1789)

at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:4838)

at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:6150)

at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteCursored(SQLServerStatement.java:1877)

at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:766)

at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:689)

at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)

at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)

at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)

at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)

at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:616)

at com.service_now.monitor.jdbc.JDBCRowSet.query(JDBCRowSet.java:64)

at com.service_now.mid.probe.JDBCProbe.doSelect(JDBCProbe.java:313)

at com.service_now.mid.probe.JDBCProbe.doQuery(JDBCProbe.java:199)

at com.service_now.mid.probe.JDBCProbe.probe(JDBCProbe.java:124)

at com.service_now.mid.probe.AProbe.process(AProbe.java:96)

at com.service_now.mid.queue_worker.AWorker.runWorker(AWorker.java:121)

at com.service_now.mid.queue_worker.AWorkerThread.run(AWorkerThread.java:20)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Any idea what I can do to fix this?

Thank you,

Heiko

Re: JDBC Connection Issue - Did not get a response from the MID Server

Michael Fry1 — Wed, 23 Oct 2019 17:21:49 GMT

Are you sure the mid-server is available and not down for maintenance or some other issue?

Re: JDBC Connection Issue - Did not get a response from the MID Server

Heiko Bllr — Thu, 24 Oct 2019 16:35:15 GMT

They are all up. I can see this error in real time in the agent log. we just increased the jdbc timeout to 30 mins without success by setting this glide.jdbcprobeloader.retry to 360.

Re: JDBC Connection Issue - Did not get a response from the MID Server

Heiko Bllr — Tue, 24 Sep 2024 16:38:14 GMT

I just realized I never commented again on my question. The root cause for the issue was, that the database query too ktoo long and then the connection basically timed out (I believe the OOTB setting is 300s if I am not mistaken). I took me a while to figure this out, only after I installed a DB client on the MID Server to run the SQL query I could see the issue and show to my client.

So in my case "Did not get a response from the MID server" did mean that the MID server could not respond back to the ECC queue with a proper result.

question Re: JDBC Connection Issue - Did not get a response from the MID Server in ITSM forum

JDBC Connection Issue - Did not get a response from the MID Server

Re: JDBC Connection Issue - Did not get a response from the MID Server

Re: JDBC Connection Issue - Did not get a response from the MID Server

Re: JDBC Connection Issue - Did not get a response from the MID Server