Impact to Real-time Integrations During Upgrades

[RitaS]

We are planning our first production instance upgrade for next month and need to evaluate impacts to users that will be on the system during the upgrade. It appears our service desk agents can continue working during the upgrade, but I'm not able to find documentation about the impacts to real-time integrations during the upgrade. We have an integration with our paging/alerting system that allows for paging on-call resources when high priority incident tickets are generated. During the upgrade, can we expect those messages to continue flowing or will there be potential delays to them? If delays should be expected, would they be for the duration of the upgrade or for a smaller window of time?

[AbdoulayeBA]

I don't have an answer for this but I am looking for the same info as we have Tenable integrated with SnowVR and wanted to know if this upgrade could mess up something.

[JC Moller]

What kind of integration baseline data do you have in your instance? How many integrations are there, and are the message types primarily SOAP or REST? What transaction volumes are you observing over 1-minute, 5-minute, 60-minute, and 24-hour intervals? This information is available on the stats.do page under the API_INT semaphore section. If you monitor inbound integration activity in the syslog_transaction table, do you frequently see session or semaphore wait times being logged? Are HTTP 429 errors appearing in the API_INT section of the stats.do page?

In our environment, we have 120 MID Servers, approximately 100 integrations, 12 application nodes, and about 175 million API transactions per month. We have never experienced any issues. Why? Because every integration is thoroughly planned, audited, and actively monitored every day. If we detect any irregularity, we take immediate and proactive action.

Below is one example of integration-generated data, specifically, response time metrics for around 100 integrations. Based on that data, I would not recommend scheduling the upgrade to begin at 18:00. We have completed about 30 upgrades across different versions and have not encountered any significant issues with inbound API integrations. Servicenow throttles the scheduler during upgrades, but we do not rely on any critical integrations triggered by the scheduler using push or pull logic.

BR, Jan