Hello,
Has anyone encountered, resolved, or have any recommendations for how to resolve this issue:
Short description: Mid Server failed to Upgrade
Failed at the "Deploy Binary File" Stage - pic attached
Things I've checked:
- Restarted service, didn't work. Still showing "Upgrading" in Status
- Also as noted below there is a warning message, "WARN: MID Server Windows Service is not running as LocalSystem and administrative rights cannot be determined" - This I am looking into this however to be fair there has been 1 successful upgrade in the past via Mid Server Upgrade History. I'll continue to research and backtrack if there were any changes to account in the meantime any ideas for resolution would be appreciated.
- AgentO.log from today 5/16:
Agent0 log:
2024-05-16T10:50:39.135-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:426] Extracting package C:\Program Files\ServiceNow MID Server def_snowmidserver_wsr22\agent\package\incoming\mid-upgrade.utah-12-21-2022__patch10a-03-18-2024_04-04-2024_0911.universal.universal.zip to C:\Program Files\ServiceNow MID Server def_snowmidserver_wsr22\agent\work\upgrade_temp\18276392314927606742
2024-05-16T10:50:40.635-0400 WARN (AutoUpgrade.3600) [MIDUpgrader:935] Unable to determine upgrade scenario with current=utah-12-21-2022__patch10-01-10-2024_01-26-2024_1356 assigned=utah-12-21-2022__patch10a-03-18-2024_04-04-2024_0911
2024-05-16T10:50:40.635-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:1015] Writing MID and wrapper PIDs to C:\Program Files\ServiceNow MID Server def_snowmidserver_wsr22\agent\conf\pids
2024-05-16T10:50:40.635-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:346] Attempt to delete file: C:\Program Files\ServiceNow MID Server def_snowmidserver_wsr22\agent\conf\DistUpgrade
2024-05-16T10:50:40.635-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:348] File delete successful
2024-05-16T10:50:40.635-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:382] Reinstall Service -- Reinstalling the service is supported only for linux OS
2024-05-16T10:50:40.635-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:501] Stopping MID server. Bootstrapping upgrade.
2024-05-16T10:50:40.635-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:502] MIDUpgrader.startUpgradeRunner(), OperationalState=UPGRADING
2024-05-16T10:51:16.809-0400 INFO (LogStatusMonitor.60) [LogStatusMonitor:50] 2024-05-16T14:51:16.809Z, stats threads: 92, memory max: 910.0mb, allocated: 266.0mb, used: 85.0mb, standard.queued: 0 probes, standard.processing: 0 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes
2024-05-16T10:52:16.737-0400 INFO (LogStatusMonitor.60) [LogStatusMonitor:50] 2024-05-16T14:52:16.737Z, stats threads: 92, memory max: 910.0mb, allocated: 318.0mb, used: 84.0mb, standard.queued: 0 probes, standard.processing: 0 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes
2024-05-16T10:53:16.713-0400 INFO (LogStatusMonitor.60) [LogStatusMonitor:50] 2024-05-16T14:53:16.713Z, stats threads: 92, memory max: 910.0mb, allocated: 255.0mb, used: 84.0mb, standard.queued: 0 probes, standard.processing: 0 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes
2024-05-16T10:54:01.034-0400 WARN (ECCQueueMonitor.1) [AMIDStateMessageFilter:68] UpgradingStateMessageFilter: Null topic or source value detected (topic: HeartbeatProbe, source: null, sys_id: 2e61030cdb5286105edc629813961983).
2024-05-16T10:54:01.034-0400 INFO (Worker-Interactive:HeartbeatProbe-2e61030cdb5286105edc629813961983) [AWorker:128] Worker starting: HeartbeatProbe
2024-05-16T10:54:01.034-0400 INFO (Worker-Interactive:HeartbeatProbe-2e61030cdb5286105edc629813961983) [HeartbeatProbe:43] Probing heartbeatprobe
2024-05-16T10:54:01.034-0400 INFO (Worker-Interactive:HeartbeatProbe-2e61030cdb5286105edc629813961983) [HeartbeatProbe:48] Finished firing the heartbeatprobe
2024-05-16T10:54:01.049-0400 WARN (Worker-Interactive:HeartbeatProbe-2e61030cdb5286105edc629813961983) [AMIDStateMessageFilter:68] UpgradingStateMessageFilter: Null topic or source value detected (topic: HeartbeatProbe, source: , sys_id: ).
2024-05-16T10:54:01.518-0400 INFO (Worker-Interactive:HeartbeatProbe-2e61030cdb5286105edc629813961983) [AWorker:136] Worker completed: HeartbeatProbe time: 0:00:00.000
2024-05-16T00:39:47.605-0400 WARN (AutoUpgrade.3600) [Log:324] ProcessRunner terminated due to interrupt java.lang.InterruptedException
2024-05-16T00:39:47.668-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:156] Added marker `C:\Program Files\ServiceNow MID Server def_snowmidserver_wsr22\agent\work\upgrade_temp\9606047952986462786` to upgrade marker file.
2024-05-16T00:39:47.668-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:290] Skip dist signal check: false
2024-05-16T00:39:47.668-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:293] assignedVersion: utah-12-21-2022__patch10a-03-18-2024_04-04-2024_0911
2024-05-16T00:39:47.668-0400 WARN (AutoUpgrade.3600) [MIDUpgrader:299] Unable to determine release version with assigned = utah-12-21-2022__patch10a-03-18-2024_04-04-2024_0911
2024-05-16T00:39:47.668-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:375] DistUpgrade file is not present.
2024-05-16T00:39:47.668-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:323] Waiting for upgrade process started signal...
2024-05-16T00:39:47.668-0400 WARN (AutoUpgrade.3600) [MIDUpgrader:327] sleep interrupted
java.lang.InterruptedException: sleep interrupted
at java.base/java.lang.Thread.sleep(Native Method)
at com.service_now.mid.dist.upgrade.MIDUpgrader.receivedDistUpgradeSignal(MIDUpgrader.java:325)
at com.service_now.mid.dist.upgrade.MIDUpgrader.waitForDistUpgradeStatus(MIDUpgrader.java:301)
at com.service_now.mid.dist.upgrade.MIDUpgrader.upgrade(MIDUpgrader.java:273)
at com.service_now.monitor.AutoUpgrade.attemptUpgrade(AutoUpgrade.java:245)
at com.service_now.monitor.AutoUpgrade.run(AutoUpgrade.java:128)
at com.service_now.monitor.AMonitor.runit(AMonitor.java:156)
at com.service_now.monitor.AMonitor$MonitorTask.execute(AMonitor.java:149)
at com.service_now.monitor.base.AMonitorTask.run(AMonitorTask.java:29)
at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
at java.base/java.util.TimerThread.run(Timer.java:506)
2024-05-16T00:39:47.668-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:375] DistUpgrade file is not present.
2024-05-16T00:39:47.668-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:323] Waiting for upgrade process started signal..
2024-05-16T00:42:18.877-0400 INFO (AutoUpgrade.3600) [WindowsAuditInstalledServices:216] verifyUpgradeServiceBlockers -- Start verifying if any MID Upgrade blocker service is running on the host machine
2024-05-16T00:42:18.877-0400 INFO (AutoUpgrade.3600) [WindowsAuditInstalledServices:225] verifyUpgradeServiceBlockers -- List of upgrade blocker services: ciscoamp
2024-05-16T00:42:18.877-0400 INFO (AutoUpgrade.3600) [WindowsAuditInstalledServices:255] verifyUpgradeServiceBlockers -- No MID Upgrade blocker service was identified on the MID Server host machine
2024-05-16T00:42:19.017-0400 WARN (AutoUpgrade.3600) [WindowsAuditInstalledServices:71] Skip populating installed services information on the instance. Unable to verify if the service table is available on the instance. The process failed with Exception:com.glide.util.GlideFamilyReleaseUncomparableException: Cannot compare if either release is unknown
2024-05-16T00:42:22.721-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:873] Result of PowerShell test:
true
WARN: MID Server Windows Service is not running as LocalSystem and administrative rights cannot be determined - This I am looking into, I've verie
2024-05-16T00:42:22.815-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:899] Deleting folder C:\Program Files\ServiceNow MID Server def_snowmidserver_wsr22\agent\package\upgradeCheck
2024-05-16T00:42:22.815-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:601] Pre-upgrade validation tests successful. Continuing with upgrade process.
2024-05-16T00:42:23.753-0400 INFO (AutoUpgrade.3600) [AutoUpgrade:262] Verifying signatures of preexisting files in download directory.
2024-05-16T00:42:23.753-0400 INFO (AutoUpgrade.3600) [MIDJarVerifier:105] Verifying digital signature of C:\Program Files\ServiceNow MID Server def_snowmidserver_wsr22\agent\package\incoming\mid-core.utah-12-21-2022__patch10a-03-18-2024_04-04-2024_0911.universal.universal.zip
2024-05-16T00:42:27.221-0400 INFO (Thread-120) [ExtensionContainer:429] Extension context pruner: Querying table ecc_agent_ext_context for list of MID Server extensions which should be running after reconnect
2024-05-16T00:42:27.221-0400 INFO (Thread-120) [SOAPSecurity:59] Getting instance ACLs for table: ecc_agent_ext_context
2024-05-16T00:42:27.784-0400 INFO (AutoUpgrade.3600) [MIDJarVerifier:105] Verifying digital signature of C:\Program Files\ServiceNow MID Server def_snowmidserver_wsr22\agent\package\incoming\mid-upgrade.utah-12-21-2022__patch10a-03-18-2024_04-04-2024_0911.universal.universal.zip
2024-05-16T00:42:28.987-0400 INFO (AutoUpgrade.3600) [Packages:322] Jre version: 11.0.16.1-sncmid1
2024-05-16T00:42:29.143-0400 INFO (AutoUpgrade.3600) [Packages:127] Packages refreshed.
2024-05-16T00:42:29.159-0400 INFO (AutoUpgrade.3600) [MIDUpgrader:223] Upgrading MID server.
