Midserver is down - i checked with the logs but couldn't find out whats the issue?

Prasanna44
Tera Contributor

‎08-01-2022 12:44 AM

    at com.service_now.monitor.base.AMonitorTask.run(AMonitorTask.java:29)
    at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
    at java.base/java.util.TimerThread.run(Timer.java:506)

08/01/22 09:32:34 (955) RefreshMonitor.65 SEVERE *** ERROR *** Failed to execute remote query: Method failed: (https://hpeglobalcsm.service-now.com/ecc_agent_property.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:32:34 (956) RefreshMonitor.65 SEVERE *** ERROR *** Failed to load remote properties: null
08/01/22 09:32:34 (962) RefreshMonitor.65 WARNING *** WARNING *** Method failed: (https://hpeglobalcsm.service-now.com/ecc_agent_property.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:32:34 (962) RefreshMonitor.65 SEVERE *** ERROR *** getRecords failed (Method failed: (https://hpeglobalcsm.service-now.com/ecc_agent_property.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403)
08/01/22 09:32:34 (962) RefreshMonitor.65 WARNING *** WARNING *** MIDRemoteGlideRecord.query failed, retrying in 10 seconds
08/01/22 09:33:14 (585) WrapperListener_stop_runner Using configuration: C:\ServiceNOW\BASF_EVT_MID_PROD\agent\config.xml
08/01/22 09:33:14 (639) WrapperListener_stop_runner Logger config: root=INFO
08/01/22 09:33:14 (641) WrapperListener_stop_runner Refreshing LoggerFactory cache
08/01/22 09:33:14 (644) WrapperListener_stop_runner Stopping MID server
08/01/22 09:33:14 (644) WrapperListener_stop_runner Main.handleStop() before shutdown, OperationalState=UP
08/01/22 09:33:25 (376) LogStatusMonitor.60 stats threads: 130, memory max: 910.0mb, allocated: 784.0mb, used: 157.0mb, standard.queued: 0 probes, standard.processing: 0 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes
08/01/22 09:33:25 (382) RefreshMonitor.65 WARNING *** WARNING *** Method failed: (https://hpeglobalcsm.service-now.com/ecc_agent_property.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:33:25 (382) RefreshMonitor.65 SEVERE *** ERROR *** getRecords failed (Method failed: (https://hpeglobalcsm.service-now.com/ecc_agent_property.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403)
08/01/22 09:33:25 (383) RefreshMonitor.65 WARNING *** WARNING *** MIDRemoteGlideRecord.query failed, retrying in 15 seconds
08/01/22 09:33:25 (386) ECCQueueMonitor.40 WARNING *** WARNING *** Method failed: (https://hpeglobalcsm.service-now.com/ecc_queue.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:33:25 (386) ECCQueueMonitor.40 SEVERE *** ERROR *** getRecords failed (Method failed: (https://hpeglobalcsm.service-now.com/ecc_queue.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403)
08/01/22 09:33:25 (386) ECCQueueMonitor.40 WARNING *** WARNING *** MIDRemoteGlideRecord.query failed, retrying in 75 seconds
08/01/22 09:33:25 (386) WrapperListener_stop_runner WARNING *** WARNING *** Method failed: (https://hpeglobalcsm.service-now.com/ecc_agent.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:33:25 (386) WrapperListener_stop_runner SEVERE *** ERROR *** getRecords failed (Method failed: (https://hpeglobalcsm.service-now.com/ecc_agent.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403)
08/01/22 09:33:25 (386) WrapperListener_stop_runner WARNING *** WARNING *** MIDRemoteGlideRecord.query failed, retrying in 10 seconds
08/01/22 09:34:04 (036) main Logger config: root=INFO
08/01/22 09:34:04 (511) WrapperStartStopAppMain Using configuration: C:\ServiceNOW\BASF_EVT_MID_PROD\agent\config.xml
08/01/22 09:34:05 (216) WrapperStartStopAppMain Logger config: root=INFO
08/01/22 09:34:05 (224) WrapperStartStopAppMain Refreshing LoggerFactory cache
08/01/22 09:34:05 (357) WrapperStartStopAppMain Welcome to the ServiceNow MID Server
08/01/22 09:34:06 (091) WrapperStartStopAppMain Attempting to load boot config file at location: C:\ServiceNOW\BASF_EVT_MID_PROD\agent\boot-config.yaml
08/01/22 09:34:06 (091) WrapperStartStopAppMain Boot configuration file not found at default location: C:\ServiceNOW\BASF_EVT_MID_PROD\agent\boot-config.yaml, providing empty configuration
08/01/22 09:34:06 (146) MIDServer Initializing MID Server
08/01/22 09:34:06 (147) MIDServer Running under Java version: 11.0.9.1-sncmid1, java PID: 24484, args: start
08/01/22 09:34:06 (330) MIDServer Running in Commercial (non-FIPS Enforced) mode
08/01/22 09:34:07 (427) MIDServer Running MID server with the following security providers:
08/01/22 09:34:07 (427) MIDServer     SUN
08/01/22 09:34:07 (427) MIDServer     SunRsaSign
08/01/22 09:34:07 (427) MIDServer     SunEC
08/01/22 09:34:07 (427) MIDServer     SunJSSE
08/01/22 09:34:07 (427) MIDServer     SunJCE
08/01/22 09:34:07 (427) MIDServer     SunJGSS
08/01/22 09:34:07 (427) MIDServer     SunSASL
08/01/22 09:34:07 (428) MIDServer     XMLDSig
08/01/22 09:34:07 (428) MIDServer     SunPCSC
08/01/22 09:34:07 (428) MIDServer     JdkLDAP
08/01/22 09:34:07 (428) MIDServer     JdkSASL
08/01/22 09:34:07 (428) MIDServer     SunPKCS11
08/01/22 09:34:07 (428) MIDServer     BCFIPS
08/01/22 09:34:07 (428) MIDServer     BCJSSE
08/01/22 09:34:07 (827) MIDServer JVM default socket factory: class com.service_now.mid.security.policy.PolicyBasedSSLSocketFactory
08/01/22 09:34:07 (828) MIDServer Creating injector...
08/01/22 09:34:07 (838) MIDServer Loaded credentials provider: com.service_now.mid.keypairs.provider.standard.StandardKeyPairsProvider
08/01/22 09:34:12 (687) MIDServer [0:00:04.819] HostUtil.getHostname: wdeluh10464162
08/01/22 09:34:16 (496) MIDServer Setting basic authentication with user BASF_EVT_MID
08/01/22 09:34:16 (506) MIDServer Using proxy host=serverproxy.basf.net, port=8080, user=
08/01/22 09:34:16 (576) MIDServer ThreadPool-Interactive started with corePoolSize: 10, maxPoolSize: 10, maximumQueueSize: 40
08/01/22 09:34:16 (583) MIDServer ThreadPool-Expedited started with corePoolSize: 20, maxPoolSize: 20, maximumQueueSize: 400
08/01/22 09:34:16 (591) MIDServer ThreadPool-Standard started with corePoolSize: 25, maxPoolSize: 25, maximumQueueSize: 500
08/01/22 09:34:16 (604) MIDServer ExtensionContainer ThreadPool started with corePoolSize: 25, maximumPoolSize: 25, maximumQueueSize: 500
08/01/22 09:34:17 (387) MIDServer MIDCredentialsConfigProvider initialized with com.service_now.mid.creds.provider.standard.StandardCredentialsProvider
08/01/22 09:34:17 (556) MIDServer WARNING *** WARNING *** Method failed: (https://hpeglobalcsm.service-now.com/file_discovery_agent_sync.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:34:17 (557) MIDServer SEVERE *** ERROR *** getRecords failed (Method failed: (https://hpeglobalcsm.service-now.com/file_discovery_agent_sync.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403)
08/01/22 09:34:17 (563) MIDServer WARNING *** WARNING *** MIDRemoteGlideRecord.query failed, retrying in 10 seconds
08/01/22 09:34:27 (570) MIDServer WARNING *** WARNING *** Method failed: (https://hpeglobalcsm.service-now.com/file_discovery_agent_sync.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:34:27 (570) MIDServer SEVERE *** ERROR *** getRecords failed (Method failed: (https://hpeglobalcsm.service-now.com/file_discovery_agent_sync.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403)
08/01/22 09:34:27 (570) MIDServer WARNING *** WARNING *** MIDRemoteGlideRecord.query failed, retrying in 15 seconds
08/01/22 09:34:42 (590) MIDServer WARNING *** WARNING *** Method failed: (https://hpeglobalcsm.service-now.com/file_discovery_agent_sync.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:34:42 (590) MIDServer SEVERE *** ERROR *** getRecords failed (Method failed: (https://hpeglobalcsm.service-now.com/file_discovery_agent_sync.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403)
08/01/22 09:34:42 (590) MIDServer WARNING *** WARNING *** MIDRemoteGlideRecord.query failed, retrying in 22 seconds
08/01/22 09:35:05 (096) MIDServer WARNING *** WARNING *** Method failed: (https://hpeglobalcsm.service-now.com/file_discovery_agent_sync.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403
08/01/22 09:35:05 (096) MIDServer SEVERE *** ERROR *** getRecords failed (Method failed: (https://hpeglobalcsm.service-now.com/file_discovery_agent_sync.do?SOAP&displayvalue=all&redirectSupported=true)HTTP/1.1 403 Forbidden with code: 403)
08/01/22 09:35:05 (097) MIDServer WARNING *** WARNING *** MIDRemoteGlideRecord.query failed, retrying in 33 seconds

Labels:
0 Helpfuls
  • 1,532 Views
6 REPLIES 6

suvro
Mega Sage
Mega Sage

‎08-01-2022 01:03 AM

Does the MID server user has mid_server role ?

0 Helpfuls

Prasanna44
Tera Contributor
In response to suvro

‎08-01-2022 02:17 AM

yes

0 Helpfuls

suvro
Mega Sage
Mega Sage
In response to Prasanna44

‎08-01-2022 05:12 AM

Was it up and running at any point ?

Best would be to create a new MID Server rather than wasting time in troubleshooting this issue. It will save you a lot of time

0 Helpfuls

Ashutosh Munot1
Kilo Patron
Kilo Patron

‎08-02-2022 03:09 AM

Go to MID Server,

Open logs check for errors,

If not then check wrapper log for initializing errors.


Do you have any service account on mid server?

0 Helpfuls