Configure rate limiting for providers

Configure rate limiting to control the traffic flow to the one_extend request by restricting the number of requests that can be made within a certain time frame.

Vorbereitungen

Role required: admin

Warum und wann dieser Vorgang ausgeführt wird

To rate limit the one_extend request, you must define the rate limit rules for the incoming requests. You can create your own rule in the One Extend Rate Limit Rules table.

Prozedur

1. On your ServiceNow instance, navigate to the One Extend Rate Limit Rules [sys_one_extend_rate_limit_rules] table and select New.
2. On the form, fill in the fields.

   Tabelle : 1. One Extend Rate Limit Rule form

   Field	Description
   Name	Name of the rate limit rule. Identify the rate limit rule with a name.
   Application	Application scope that must be set to Global.
   Resource	Resources to select for your rate limit rule. Use one of the following resources for your rate limit rule: LLM Provider: For the LLM Provider, select the generative AI provider mapping. Capability Definition: For the Capability Definition, select the One Extend Capability Definition. All Capabilities: Applies to all providers and all capability definitions.
   Rule Type	Rule type for the rate limit rule. Define the rule type using the following options: Instance: Accumulates data from all the users of that instance. User: Keeps the count of the requests per user.
   Request limit per hour	Maximum number of requests to be received per hour.
   Active	Status of the rate limit rule.

3. Select Submit.