Perfomance Issues - Slow - On Prem
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-01-2023 03:01 AM - edited 11-01-2023 03:12 AM
We have almost weekly performance issues on our ServiceNow instance (slow and you have to use F5 a lot to load pages), restarting the glide is then the only solution.
There are no issues with memory, storage or CPU on the servers. There are no network issues.
After investigation of the web server, database and operating systems we have not found any reasons for the lag.
We see the following repeating messages in the "wrapper log":
INFO | jvm 2 | 2023/11/01 00:07:51.941 | 2023-11-01 00:07:51 (855) WebSocket background processing WebSocket Error, Address: null
INFO | jvm 2 | 2023/11/01 00:07:51.941 | 2023-11-01 00:07:51 (913) https-jsse-nio2-8443-exec-22 SEVERE Error reading request, ignored
INFO | jvm 2 | 2023/11/01 00:07:51.941 | 2023-11-01 00:07:51 (855) WebSocket background processing WebSocket Error, Address: null
INFO | jvm 2 | 2023/11/01 00:07:51.941 | java.io.IOException: java.io.IOException: Channel is in closing state.
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:327)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:262)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsSession.sendCloseMessage(WsSession.java:816)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsSession.doClose(WsSession.java:704)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsSession.doClose(WsSession.java:666)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsSession.checkExpiration(WsSession.java:1045)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsWebSocketContainer.backgroundProcess(WsWebSocketContainer.java:1089)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.BackgroundProcessManager.process(BackgroundProcessManager.java:91)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.BackgroundProcessManager.access$000(BackgroundProcessManager.java:32)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.BackgroundProcessManager$WsBackgroundThread.run(BackgroundProcessManager.java:140)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | Caused by: java.io.IOException: Channel is in closing state.
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.SecureNio2Channel$FutureWrite.<init>(SecureNio2Channel.java:844)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.SecureNio2Channel$FutureWrite.<init>(SecureNio2Channel.java:835)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.SecureNio2Channel.write(SecureNio2Channel.java:923)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper.doWrite(Nio2Endpoint.java:1173)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:718)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking(SocketWrapperBase.java:671)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper.flushBlocking(Nio2Endpoint.java:1216)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.SocketWrapperBase.flush(SocketWrapperBase.java:661)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.server.WsRemoteEndpointImplServer.doWrite(WsRemoteEndpointImplServer.java:98)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.writeMessagePart(WsRemoteEndpointImplBase.java:512)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.websocket.WsRemoteEndpointImplBase.sendMessageBlock(WsRemoteEndpointImplBase.java:314)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | ... 9 more
INFO | jvm 2 | 2023/11/01 00:07:51.941 |
INFO | jvm 2 | 2023/11/01 00:07:51.941 | 2023-11-01 00:07:51 (913) https-jsse-nio2-8443-exec-22 SEVERE Error reading request, ignored
INFO | jvm 2 | 2023/11/01 00:07:51.941 | java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "setContextClassLoader")
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at java.security.AccessController.checkPermission(AccessController.java:884)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at java.lang.Thread.setContextClassLoader(Thread.java:1471)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.catalina.core.StandardContext.bind(StandardContext.java:5864)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.catalina.core.AsyncContextImpl.fireOnComplete(AsyncContextImpl.java:101)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.coyote.AsyncStateMachine.asyncPostProcess(AsyncStateMachine.java:284)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.coyote.AbstractProcessor.asyncPostProcess(AbstractProcessor.java:200)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:81)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRun(Nio2Endpoint.java:1615)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at java.lang.Thread.run(Thread.java:745)
INFO | jvm 2 | 2023/11/01 00:07:51.941 |
INFO | jvm 2 | 2023/11/01 00:07:51.941 | 2023-11-01 00:07:51 (913) https-jsse-nio2-8443-exec-22 Encountered a non-recycled request and recycled it forcedly.
INFO | jvm 2 | 2023/11/01 00:07:51.941 | org.apache.catalina.connector.CoyoteAdapter$RecycleRequiredException
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.catalina.connector.CoyoteAdapter.checkRecycled(CoyoteAdapter.java:546)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.coyote.http11.Http11Processor.recycle(Http11Processor.java:1587)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.coyote.AbstractProtocol$ConnectionHandler.release(AbstractProtocol.java:1021)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:989)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRun(Nio2Endpoint.java:1615)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
INFO | jvm 2 | 2023/11/01 00:07:51.941 | at java.lang.Thread.run(Thread.java:745)
We are currently running Tokyo Patch 8, but this has always been a problem in previous versions as well.
Do you guys recognize this? Do you have any idea where to look for this?
Full wrapperlog added as attachment.
Thanks in advance!
0 REPLIES 0
