OAuth Bearer Token Near Expiration

[techpriest001]

I've been working with my developer instance and exploring the API.   I am successful at making API calls using the OAuth integration with the "password" grant type.   But I've noticed something that seems less-than-optimal in regards to the access token and refresh token.   In my organization with our current ticket/incident system (not ServiceNow), we frequently make calls into that system to generate tickets, and we also drive automated workflows by polling our system for tickets/requests that can be serviced through automation. I expect we'll be doing the same with ServiceNow.   What I am concerned about is failed API calls due to calls being made near the expiration threshold of the access token's lifetime.

Please consider this example:

1. One of our internal process begins polling ServiceNow for requests that can be automated.
2. We initiate our first call into ServiceNow and we get our authorization token with a lifetime of 1799 seconds.
3. Due to unfortunate timing, another process polls ServiceNow 1798'ish seconds later.
4. We try to make a call into ServiceNow, but by the time the call launches and connects, our token is expired and our API call is rejected.

I have spent nearly 14 years doing API integrations, and the last 3 or so years working intimately with OAuth-protected APIs.   We could have any number of processes making calls into ServiceNow, and any number of them could land right at the access token's expiration threshold.   With other APIs in other systems, we eliminate this issue by looking at the remaining lifetime of the access token.   If the access token is within X seconds of expiration, we will preemptively refresh the token if the API allows it and the refresh token itself isn't expired, or we will initiate a new "login" to obtain a new access token and refresh token pair.

Unfortunately, ServiceNow's OAuth implementation seems like it makes it impossible to prevent this type of service interruption blip.   The issued access token does not include the absolute expiration date/time of the token.   The expiration time cannot be inferred because the creation time of the access token is not returned either.   Internally we can track when we requested the token and estimate the expiration time, but making the "refresh token" call simply returns the existing access token when we're too early.   Similarly, if we try to obtain a completely new access token early, we also get back the exact same existing access token we started with.

I guess I'm not sure what value the refresh token even provides, because we can't avoid "threshold expiration" by preemptively refreshing the token early.   Instead, I think we're going to have to create some kind of wasteful retry loop to ensure that if a call is made with a token about to expire, then we'll retry the operation after doing a new login attempt to obtaining an entirely new token. For this retry workaround, I wouldn't even bother with the refresh process because the calling code on our side won't know if the refresh token is also expired.

Has anyone else run into this issue or see something that I'm missing?   My solution would be for ServiceNow to issue tokens that include the expiration of the access token in UTC and also the expiration of the refresh token in UTC.   And ideally, the refresh process should always return a new token.

[scottl]

Did you end up finding a solution to this issue, as I have discovered the same issue.  It seems SN only considered OAuth for end-user websites, with the manual "accept|deny" end-user experience. While ignoring service integration endpoints for 3rd party access!

[techpriest001]

No I didn't.  I didn't think the issue I reported was ever truly understood, and util you replied I worried that no one understood what I wrote. In our implementation, some sort of wrapper API was put over the native API, and the wrapper API requires HTTP basic auth. So there wasn't really any driver for me to pursue this.

[scottl]

From what I've observed, when the access_token has expired, the record is deleted from the oauth credential table. So the scripted service has to check that if Auth has not been granted, it needs to generate another access_token, with the client_id, client_secret and refresh_token through the oauth_token.do path.  I believe the same goes for the refresh_token, however it would have to go through the oauth_auth.do URL, with the client_id and client_secret... and also the local user account with the username and password.  However, this kinda defeats the point of Oauth, if one has to supply the 3rd party with the local user account of the username & password, as that is passed no differently as Basic Auth, isn't it?

So to sum up, 3rd party services need alot of conditional testing on the Auth connection prior, if it fails, to then gain access (re-create all tokens) to the data while also having the local user account details stored at their end to achieve a authorized connection.  Thus making testing for token time expiry redunant based on what SN sends to the client.

[rick_gwu]

Unfortunately a "retry loop" is the only way to go.

[rick_gwu]

Unfortunately, a "retry loop" is the only way to go.