Performance issues across entire system all users

Katie A
Mega Guru

We are having some very impactful performance issues in our instance. We are concerned because we are on the verge of populating our CMDB with multiple data sources -- but we aren't comfortable proceeding considering the system performance is already terrible. Most of the time pages are just very slow to load, but other times the page does not load at all and the browser crashes.

The issue affects ALL users in ALL areas of the system. The issue is affecting production as well as dev and test instances.

We are seeing client transaction times around 15 seconds, some as high as 20 seconds.

I opened a ticket with support but they gave us the run-around with insufficient answers, such as checking homepage refresh times. They were dismissive and tried to mark the incident as resolved without giving us any real solutions.

They checked out the health of our nodes and explained that everything seemed normal. We moved data centers about a month ago and we have requested moving back to the old data center to see if that might have been the cause of the issue.

Still, we are skeptical and extremely unhappy.

The issue cannot be homepage refresh times since the same issue is impacting both Dev and Test where there is only ONE user working there on a daily basis.We are a small company and we only have about 40 users in the system.There is no way that homepage refresh times are causing such response issues.

I checked all of the performance graphs on CPU, JVM memory, SQL transactions, etc. I don't see anything obvious in those graphs to indicate a problem.

Attached is a screenshot of the client response times which is very slow.

We are on Fuji Patch 10. The issue started about a month ago and has gotten worse in the past few weeks.

Has anyone experienced similar issues? We are not sure how to proceed considering the lack of real help from support.

33 REPLIES 33

gemini17
Kilo Guru

Hello Kda:



If the performance is affecting all users, it could be ISP third party services, for your instance, when you log in multiples ISPs and reaches SNOW.   It could be interruption in any one of the ISPs.   We had a similar situation.   We raised an in incident in HI, and resolved it.   You should the same immediately.


Hope this is helpful.


P.S. Hit it correct as you see the impact.


Kind regards


Danny Raj


Hi Danny thanks yes we opened up a ticket in HI but they did not give us any real answers, except to blame it on homepage refresh times which is not a relevant answer. I hope they look at it more and give us a better solution.


HugoFirst
Kilo Sage

Turn on the response time indicator.


See Response Time Indicator - ServiceNow Wiki  



That will break you response time down into 3 categories:


1. Network


2. Server


3. Browser



Once you see those numbers, you will have a place to look.


Hi Steve, Thanks we have been collecting those from our users, an example is attached. In most cases the Server   is the slowest response and there have been a few examples where the browser was slowest. The Network is always below 10ms.



Screen Shot 2016-03-16 at 12.51.12 PM.pngScreen Shot 2016-03-16 at 12.48.12 PM.png