Recently, several reports in Power BI started failing. The reports were retrieving data via ODBC and were using " SELECT * " queries.
This ODBC connection stopped working, apparently because it was generating too many requests.
As an alternative, we are planning to set up a PostgreSQL database on a VM, where we will query all the data via REST API and insert it into this database.
I’d like to know what are the best practices for querying the REST API — such as how to identify rate limits, how many requests are performed per query, etc. — in order to avoid the same issue that occurred with ODBC. I’ll be querying the APIs using Python.
While researching the documentation, I’ve already found some recommendations, such as:
Using sysparm_offset for pagination (I’m thinking of using around 2500 records per page);
Querying only records that were modified, based on the sys_updated_on column;
Filtering only necessary columns;
Scheduling the queries during off-peak hours.
Can anyone confirm if this solution is safe — meaning it won’t trigger “too many requests” errors, slow down ServiceNow, or lead to the same issue we had with ODBC?
The tables I need are:
sc_request
incident
problem
task_sla
task
core_company
ast_contract
