MID server not responding

andy_trayler
Giga Contributor

I am currently developing on EUREKA (prior to the move to FUJI) and I am experiencing problems with the MID server not returning responses to JDBC SQL requests agains an SQL server DB. The response that I'm seeing on the Data Source importation is MID server did not respond (within the 300 sec time period). I am seeing the ecc queue being to accumulkate items in the "processing" state and eventually nothing is handler by the MID server. Any one have any suggestion?

9 REPLIES 9

tony_barratt
ServiceNow Employee
ServiceNow Employee

Hi Andy,



Could you check the network connectivity from the MID-Server host towards the MS-SQL DB with telnet hostname port number? Default port number is 1433.


Maybe you have already done that.



The MS-SQL admin could check in the logs for failed logins.


https://support.microsoft.com/en-us/kb/2187161?wa=wsignin1.0
It mentions "In SQL Server 2008 and later, auditing failed login attempts is enabled by default."




Sometimes counter intuitive installs are deployed, such as SQL Server 2008 and 2012 installed on the same server


You could also check with the MS-SQL admin what the connection string should be to connect to your target db.


It is also good to know what version of MS-SQL you need to connect to.



I expect you have checked the MID-Server is up and running, have you checked the version to ensure it has the same build stamp of the instance (as seen in stats.do output)?


Best Regards



Tony







You could also pull back the MID-Server wrapper and agent logs to look for clues..


andy_trayler
Giga Contributor

Hi Guys,



thanks for the input. I have done some addiitonal investigation and I believe that the problem may be related to Semaphores. I see in the MID Server log that as part of a JDBC request coming from SN to the MID Server, I am seeing a request for a semaphore:



7/09/15 13:23:19 (269) Probe: JDBCProbe:af8a93e26f62f9007412d667ad3ee4c2 Worker starting: JDBCProbe source: af8a93e26f62f9007412d667ad3ee4c2


07/09/15 13:23:19 (821) Probe: JDBCProbe:af8a93e26f62f9007412d667ad3ee4c2 DEBUG: Using connection string 'jdbc:sqlserver://HOSTNAME:50131;instanceName=MSSQL;databaseName=iQDaxxxxxx' and driver: com.microsoft.sqlserver.jdbc.SQLServerDriver


07/09/15 13:23:19 (822) Probe: JDBCProbe:af8a93e26f62f9007412d667ad3ee4c2 DEBUG: JDBC connected


07/09/15 13:23:19 (822) Probe: JDBCProbe:af8a93e26f62f9007412d667ad3ee4c2 Waiting for connection semaphore



When thing are working fine: I see the following log line in the MID server log:



07/09/15 13:18:19 (609) Probe: JDBCProbe:5f9957e26f62f9007412d667ad3ee4ea Obtained connection semaphore



When import set jobs start to fail, I do not see this line. The import jobs continue to fail from that point on. ECC queue has a number of outbound jobs with a state of processing that are not being cleared down.



Is it possible to increase the number of semaphores that are available to test this as a scenario.



Additional Item: just check for otehr possibel items related to this and this sounds suspiciously like:


ttps://hi.service-now.com/kb_view.do?sysparm_article=KB0547566


Hi Andy,



KB0547566 if I had to guess I would say this is not related to your issue.


Raises the question of what version of Eureka are you on?



Seems like your MID Server is busy, or adjacent components are.


Question that spring to mind are:


Do you have a dedicated MID server host for the instance you are working on? Or are there multiple MID-Servers on the same Windows host? Or is it linux? Do you have a cluster of MID Servers?


Is   CPU RAM usage OK for the host?



Could it be that your instance is doing lots of importing, via the MID Server, with synchronous transforms that is causing pending actions to back up and time out?



Just thinking out loud. Input from other persons probably helpful 🙂



Best Regards



Tony