Best approach for incremental data import using REST Integration Hub Data SourceIncremental REST API

[shashitaya]

Hi everyone,

I have an external system that acts as the master source for our data. The data is CREATED/updated daily, and I need to integrate it into ServiceNow.

The external API returns the data in paginated responses. My requirement is to retrieve and process only the data that has changed since the last successful import/transform, rather than processing the entire dataset every day.

I NEED to use the ServiceNow REST Integration Hub Data Source Concept to retrieve the data, followed by an Import Set and Transform Map to load the data into my target table.

I’d like to understand the recommended approach for implementing this.

Specifically:

• How can I maintain the last successfully processed timestamp/value in ServiceNow?
• How can I pass that value to the external API so that it returns only records changed after that point?
• Since the API response is paginated, what is the recommended way to handle pagination across multiple API calls while maintaining the incremental-load logic?
• How can I ensure that the "last processed" value is updated only after all pages have been successfully retrieved and transformed?
• Is the standard REST Data Source → Import Set → Transform Map → Target Table pattern suitable for this scenario, or is there a better ServiceNow integration pattern?

I’d especially appreciate any guidance or examples from someone who has implemented incremental REST API imports with pagination using ServiceNow Integration Hub.

Thanks in advance!