Controlling session utilization from inbound integrations (sticky sessions)

[GTSPerformance]

Overview

This guide is written by the ServiceNow Technical Support Performance team (All Articles). We are a global group of experts that help our customers with performance issues. If you have questions about the content of this article we will try to answer them here. However, if you have urgent questions or specific issues, please see the list of resources on our profile page: ServiceNowPerformanceGTS.

Its widely understood that the ServiceNow platform, at a very high level, is comprised of Java Virtual Machines (JVMs or, as we call them, application nodes) sitting on top of a relational database. For the platform to perform well its absolutely key that application nodes are in good health - if nodes start to slow down then everything executing on the platform will suffer (i.e. end user transactions, integrations, and scheduled jobs) and ultimately your users will start to complain.

The most common application node issue we see is memory contention, or more accurately, application nodes spending more and more time running garbage collection (which pauses the entire node each time it executes causing throughput to drop off). There are any number of ways in which memory contention can be triggered (i.e. bad custom code, product defects, instance misconfiguration and so on) but in this article we want to concentrate on one area which tends to be ignored - inbound integrations and the sessions they create.

If we think about a user (or integration) interacting with a ServiceNow instance one of the first things which happens is that user will get a session created. Sessions are used for all kinds of different things but in reality each session is an object which sits in memory on one specific application node until it is destroyed (either by the corresponding user logging out or being inactive for a certain period of time). Given that sessions sit in memory it follows that:

• Having a large number of sessions on a single application node can cause memory contention (as, in total, session objects will use large volumes of memory which can't be used for anything else)
• Having large session 'churn' (i.e. sessions constantly being created and destroyed) can also cause memory contention (as for every session there will be an area of memory which needs to be cleaned up via garbage collection before that memory can be reused)

In terms of real end users there is generally a one to one relationship between that user and a session however, depending on how your integrations are configured, you might have a single integration which creates (and destroys) huge numbers of sessions thereby adding to memory churn / pressure. This issue can be seen whether you are using SOAP or REST however the remainder of this document focuses on REST (however all steps are equally applicable to either protocol).

To describe this in more detail lets consider a real customer scenario.

• In one of our larger customer organizations the majority of their user base doesn't access ServiceNow directly but instead hit a series of custom portals which sit in front of their instance. As they interact with these portals data is pushed to / pulled from ServiceNow via a REST integration (using a set of locally authenticated ServiceNow user accounts)
• The customer estimated that a single end user 'interaction' with a portal (i.e. viewing a record) could result in approximately 10 REST transactions being sent from the portal to the instance
• The way in which the customer had configured their REST integrations, however, meant that every single transaction executed in its own session (i.e. for every transaction a session object had to be created then subsequently destroyed)
• During peak periods integrations transactions were creating up to 600000 session per hour - if we estimate that each session object is ~20Kb in size just handling the creation and destruction of these sessions generates a churn in memory of > 11Gb per hour

Given that this customer was seeing memory contention on their application nodes and one of the more significant contributors was session churn we postulated that if their integrations could use a single session for each user 'interaction' (i.e. approximately 10 transactions) instead of a session per transaction the overhead of session creation (and destruction) for these integrations could be reduced by a factor of 10 - which would be a very good thing - but how do we make this happen?

As it turns out this is actually pretty straightforwards and is all down to cookies. To explain:

• Whenever a user or integration executes a transaction within the instance the instance will send back a response (i.e. some kind of payload or even just a status code)
• In the header of this response there are certain cookies which describe how / where the transaction was executed - the most important (in the context of this discussion) are:
  • BIGipServerpool_[instance name]: Describes the application node which executed the transaction (and therefore where the session object was created)
  • JSESSIONID: Contains the 'name' (32 character string) of the session object used to execute the transaction
• If the user (or integration) includes these cookies in the next transaction it sends to the instance then:
  • The load balancer will use the BIGipServerpool_[instance name] cookie to route the transaction to the same node that executed the prior transaction
  • The application node will use the JSESSIONID cookie to execute the transaction on the existing session with this name (if it exists - if it doesn't a new session will be spun up as required)

This means that by including these cookies in the header of requests from integrations we can help them re-use existing sessions and avoid 'session churn'

The best way to demonstrate this is via a quick Python script (shown below). In summary this script:

• Executes a REST get request against a scripted REST endpoint in a ServiceNow instance 10 times
• On receiving the response from the first request it extracts the BIGipServerpool and JSESSIONID cookies and includes these in all 9 remaining REST requests

# Import requests module
import requests

# Variable to hold cookies
cookieString = ""

# Execute 10 iterations of REST transaction

for i in range(10):

# Check whether we need to set cookies for the next request
   if (i > 0):
      headers= {"Content-Type":"application/json","Accept":"application/json","cookie":cookieString}
   else:
      headers = {"Content-Type":"application/json","Accept":"application/json"}

# Do the HTTP request
response = requests.get("https://xxx.service-now.com/api/snc/cookie_test", auth=("xxx", "xxx"), headers=headers )

# Save cookies to a dictionary
cookieDict = response.cookies.get_dict()

# Build the cookies string after the first response if we are using a single session
if (len(cookieString) == 0):
   for j in cookieDict:
      if ((j == "JSESSIONID") or (j.startswith("BIGipServerpool_"))):
         if (len(cookieString) > 0):
            cookieString += ";"
         cookieString += str(j) + "=" + str(response.cookies[j])

If we execute this script against a simple instance with two primary application nodes then we see that, as expected, all 10 transactions execute on the same application node using the same session, i.e.:

Node 1:

2023-09-11 08:15:54 (206) API_INT-thread-1 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=912cf1581b51 *** Start #7326 /api/snc/cookie_test, user: admin
2023-09-11 08:15:55 (822) API_INT-thread-4 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=a92cb1541b91 *** Start #7327 /api/snc/cookie_test, user: admin
2023-09-11 08:15:57 (386) API_INT-thread-1 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=8e2c75181b51 *** Start #7328 /api/snc/cookie_test, user: admin
2023-09-11 08:15:58 (942) API_INT-thread-4 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=e22cb9d01b91 *** Start #7329 /api/snc/cookie_test, user: admin
2023-09-11 08:16:00 (551) API_INT-thread-1 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=f62c75901b91 *** Start #7330 /api/snc/cookie_test, user: admin
2023-09-11 08:16:02 (244) API_INT-thread-4 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=1f2c3dd81b51 *** Start #7331 /api/snc/cookie_test, user: admin
2023-09-11 08:16:03 (844) API_INT-thread-1 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=af2cf9d41b51 *** Start #7332 /api/snc/cookie_test, user: admin
2023-09-11 08:16:05 (425) API_INT-thread-4 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=483c79101b91 *** Start #7333 /api/snc/cookie_test, user: admin
2023-09-11 08:16:06 (985) API_INT-thread-1 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=1c3c31541b91 *** Start #7334 /api/snc/cookie_test, user: admin
2023-09-11 08:16:08 (536) API_INT-thread-4 5D2CF1581B51F1102CE7BA63CC4BCBF8 txid=303c35dc1b51 *** Start #7335 /api/snc/cookie_test, user: admin

Node 2:

No transactions

If we only pass the BIGipServerpool cookie but not the JSESSIONID cookie then all transactions will execute on the same node (as the load balancer uses the BIGipServerpool cookie to direct all transactions to a consistent node) but use a different session for each transaction:

Node 1:

2023-09-11 08:17:47 (112) API_INT-thread-2 7C9CB9D01B91F1102CE7BA63CC4BCBA2 txid=b09cb9d01b91 *** Start #7341 /api/snc/cookie_test, user: admin
2023-09-11 08:17:48 (683) API_INT-thread-1 D19CB9D41B51F1102CE7BA63CC4BCB9D txid=199cb9d41b51 *** Start #7342 /api/snc/cookie_test, user: admin
2023-09-11 08:17:50 (253) API_INT-thread-2 E19CF91C1B51F1102CE7BA63CC4BCBE3 txid=299cf91c1b51 *** Start #7343 /api/snc/cookie_test, user: admin
2023-09-11 08:17:51 (823) API_INT-thread-1 4E9CB9941B51F1102CE7BA63CC4BCB54 txid=829cb9941b51 *** Start #7344 /api/snc/cookie_test, user: admin
2023-09-11 08:17:53 (386) API_INT-thread-2 5A9C75D41B51F1102CE7BA63CC4BCB2D txid=9e9c75d41b51 *** Start #7345 /api/snc/cookie_test, user: admin
2023-09-11 08:17:54 (984) API_INT-thread-1 3A9CF9D41B51F1102CE7BA63CC4BCBA1 txid=7e9cf9d41b51 *** Start #7346 /api/snc/cookie_test, user: admin
2023-09-11 08:17:56 (530) API_INT-thread-2 0B9CBD5C1B51F1102CE7BA63CC4BCB56 txid=4f9cbd5c1b51 *** Start #7347 /api/snc/cookie_test, user: admin
2023-09-11 08:17:58 (125) API_INT-thread-1 E39CB1581B51F1102CE7BA63CC4BCB5F txid=2b9cb1581b51 *** Start #7348 /api/snc/cookie_test, user: admin
2023-09-11 08:17:59 (689) API_INT-thread-2 3B9C35941B91F1102CE7BA63CC4BCB05 txid=7f9c35941b91 *** Start #7349 /api/snc/cookie_test, user: admin
2023-09-11 08:18:01 (288) API_INT-thread-1 90ACF5D01B91F1102CE7BA63CC4BCB49 txid=d4acf5d01b91 *** Start #7350 /api/snc/cookie_test, user: admin

Node 2:

No transactions

If we only pass the JSESSIONID cookie but not the BIGipServerpool cookie the transactions run across both nodes - where they hit the node containing the session specified by the JSESSIONID cookie they use that session but on the other node they use separate sessions each time:

Node 1:

2023-09-11 08:11:01 (479) API_INT-thread-1 66FA7194DB95F110899A3EC3E2961943 txid=8a0bf518db55 *** Start #8984 /api/snc/cookie_test, user: admin
2023-09-11 08:19:43 (857) API_INT-thread-4 690D3110DB95F110899A3EC3E296195D txid=ad0d3110db95 *** Start #9007 /api/snc/cookie_test, user: admin
2023-09-11 08:19:45 (441) API_INT-thread-2 690D3110DB95F110899A3EC3E296195D txid=f90d711cdb55 *** Start #9008 /api/snc/cookie_test, user: admin
2023-09-11 08:19:50 (279) API_INT-thread-4 690D3110DB95F110899A3EC3E296195D txid=4b0df55cdb55 *** Start #9009 /api/snc/cookie_test, user: admin
2023-09-11 08:19:51 (844) API_INT-thread-2 690D3110DB95F110899A3EC3E296195D txid=ef0d7d14db95 *** Start #9010 /api/snc/cookie_test, user: admin
2023-09-11 08:19:55 (067) API_INT-thread-4 690D3110DB95F110899A3EC3E296195D txid=9c1df598db95 *** Start #9011 /api/snc/cookie_test, user: admin

Node 2:

2023-09-11 08:19:47 (094) API_INT-thread-4 020D35DC1B51F1102CE7BA63CC4BCB28 txid=460d35dc1b51 *** Start #7357 /api/snc/cookie_test, user: admin
2023-09-11 08:19:48 (704) API_INT-thread-3 EA0DF5D01B91F1102CE7BA63CC4BCB49 txid=220df5d01b91 *** Start #7358 /api/snc/cookie_test, user: admin
2023-09-11 08:19:53 (442) API_INT-thread-4 F70D39981B51F1102CE7BA63CC4BCBC4 txid=3f0d39981b51 *** Start #7359 /api/snc/cookie_test, user: admin
2023-09-11 08:19:58 (214) API_INT-thread-3 7C1DF9101B91F1102CE7BA63CC4BCB91 txid=b01df9101b91 *** Start #7360 /api/snc/cookie_test, user: admin

If we pass neither cookie then each transaction executes on an arbitrary node and session (i.e. similar to how our customer had things configured):

Node 1:

2023-09-11 08:21:59 (082) API_INT-thread-1 568D71101B91F1102CE7BA63CC4BCB5F txid=9a8d71101b91 *** Start #7366 /api/snc/cookie_test, user: admin
2023-09-11 08:22:00 (650) API_INT-thread-4 BE8DF95C1B51F1102CE7BA63CC4BCB15 txid=f28df95c1b51 *** Start #7367 /api/snc/cookie_test, user: admin
2023-09-11 08:22:03 (784) API_INT-thread-1 2B8DF9981B51F1102CE7BA63CC4BCB4A txid=6f8df9981b51 *** Start #7368 /api/snc/cookie_test, user: admin
2023-09-11 08:22:08 (477) API_INT-thread-4 E89DF1581B51F1102CE7BA63CC4BCBFA txid=209df1581b51 *** Start #7369 /api/snc/cookie_test, user: admin
2023-09-11 08:22:10 (045) API_INT-thread-1 859DFDD81B51F1102CE7BA63CC4BCB80 txid=c99dfdd81b51 *** Start #7370 /api/snc/cookie_test, user: admin

Node 2:

2023-09-11 08:19:56 (629) API_INT-thread-2 690D3110DB95F110899A3EC3E296195D txid=ac1df1d0db95 *** Start #9012 /api/snc/cookie_test, user: admin
2023-09-11 08:22:02 (221) API_INT-thread-3 5F8D3910DB95F110899A3EC3E2961909 txid=938d3910db95 *** Start #9019 /api/snc/cookie_test, user: admin
2023-09-11 08:22:05 (355) API_INT-thread-1 CC9DF994DB95F110899A3EC3E296196C txid=049df994db95 *** Start #9020 /api/snc/cookie_test, user: admin
2023-09-11 08:22:06 (918) API_INT-thread-3 609DF5D4DB95F110899A3EC3E29619E8 txid=a49df5d4db95 *** Start #9021 /api/snc/cookie_test, user: admin
2023-09-11 08:22:11 (602) API_INT-thread-1 AD9DF198DB95F110899A3EC3E29619DD txid=e19df198db95 *** Start #9022 /api/snc/cookie_test, user: admin

So we've already described why it might make sense to do this but there are other benefits - for example you can potentially start to cache data in your session objects so that this data can be re-used by subsequent transactions using the same session (potentially providing a performance benefit). As with everything, however, there are also some down sides so lets talk about these quickly:

• As demonstrated the BIGipServerpool cookie tells the load balancer where (i.e. which application node) to send the transaction. This means that any transactions which are sent with the same value in this cookie will all execute on a single node. If you have an integration which is 'busy' (in terms of transactional volume) you almost certainly wouldn't want to use just one node as the integration could overload that node whilst other nodes effectively sit idle (in terms of this integration). In this customers case this wasn't an issue as we were only recommending sending transactions from a single user 'interaction' to the same node / session (approximately 10 transactions at a time)
• Again we've already seen that the JSESSIONID cookie describes which session should be used to execute a transaction. The ServiceNow platform has strict rules around what an individual session can do, i.e. by default:
  • A single session can only queue a maximum of 10 transactions in a single nodes transaction queue (known as 'max waiters')
  • A single session can only execute 1 transaction at a time (known as transaction concurrency or session synchronization)

This means that by using a single session you force execution of your integrations transactions to be serialized on the application node (which could cause degradation in integration performance) and enforce a maximum number of transactions the integration can send at once (if it tries to queue more than the default of 10 transactions these additional transactions will be rejected / will not execute). Again in our customers scenario this didn't matter as individual transactions were generated serially by the integration and were fast

Hopefully this article explains the relationship between integrations and sessions and how, with some clever integration design, you can avoid performance issues and build truly scalable integrations.