How can we add a rate limit, or throttle, to outbound HTTP connections?

[Shawn Gienow]

I've created an outbound HTTP connection, that POSTs to Oracle. The issue is that Oracle cannot handle the amount of traffic that our lower environments can generate.

Can we add some kind of rate limit, or throttle the outbound HTTP requests?

[Eric W]

The simplest, safest approach to this could be a Retry policy (servicenow.com)

This allows both systems to operate as designed, supporting the occasional need to back off when the remote partner is under load.

If that doesn't sufficiently address the issue, then designing a safe rate limiter depends on the processes (what and how) generating the workload.

Artificially rate-limiting requests often brings more problems than it solves.

If the load generator is an artificial source, simply generating workload (e.g. for the purposes of load testing) then it would be better to back that off at the source (rather than at the point of invocation of the connection).

If the workload is all legitimate and needs to be processed, then the retry policy is probably still cleanest -- it will all get processed in time.

If the workload is all legitimate and needs to be processed, and the remote partner can never catch up, then there is likely a larger capacity or scheduling mismatch that needs to be addressed.

[Shawn Gienow]

Yeah, that's pretty much what we came up with... its not even the retry that's an issue -- All the data is "getting there," but the process on the other side cannot handle what SN can send it 🙂 

I suggested a buffer of some sort for them, since they're not even sending us failure messages or anything similar back, so we can't implement a retry if they're saying "it worked fine," and did not.

Thank you!