Slowness detected in On-Prem instance

[SagarJ957172966]

Dear Team,

We are on ServiceNow On-Prem instance with 3 App servers and total 15 nodes across the servers. 

We have seen slowness while accessing ServiceNow application. 

Please refer to below details;

OS Version :

NAME="Red Hat Enterprise Linux"
VERSION="8.10 (Ootpa)"

Database : Oracle 19c in cluster environment

I would appreciate if anyone can help me to resolve this issue.

[jeffreyluto]

Network latency ( this is from the self-hosted FAQs  ServiceNow)

ServiceNow's standard for network latency between App -> DB servers is a minimum ICMP ping time of less or equal to.250ms. Many self-hosted environments will deploy network gear inline between the App -> DB servers which at time incur network latency. Some of these devices might be firewalls, IPS/IDS appliances or even proxy servers. Some customers have found that installing the App servers in the same network (on the same LAN) has solved any latency issues.

It's recommended your network latency is monitored to ensure the latency stays at or below .250ms. It's also highly recommended to have in place monitoring for packet loss. We've seen sever performance impacts to environments where switch gear starts failing and packet loss is one of the side effects.   

Here are some things I would suggest and at times worked for us.

-Consider putting DB and App in the same subnet

-Consider creating specific routes or QoS for this traffic

-Consider creating a different route via another vNIC than the inbound from the load balancer?

-Double-check Iptables/Firewalld

-beware of Network proxies

-Run traceroute and ping to see where and how long traffic take and adjust