How to handle huge results given by the JDBC step in Flow designer.

[Ashuthosh_s_p]

 First of all I am new to ServiceNow so flow designer is a new thing for me.Basically, I have a external Database through which i am fetching the data .This is done through the JDBC Step where I use my connection alias to connect the database and also the SQL  section to write my SQL query. Using that ResultSet, I am inserting into a table  through Script Step where the inserting is also happening perfectly

The Issue that I am facing is that I cant able to process large result from the JDBC step and use it in Script Step. Lets say I have 50000 data and JDBC step is able to process only 1000 data as it fits under its payload (ie 10MB) but I am unable to reprocess those remaining data.

How can i make batch out of those 50000 data so that it can be used in my JDBC and Script Step

(PS we don't have data stream currently so that option is not present)

[Community Alums]

Hi @Ashuthosh_s_p ,

I can think of using pagination to get a stream of data in chunks.

https://docs.servicenow.com/bundle/xanadu-integrate-applications/page/administer/integrationhub/conc...

[Ashuthosh_s_p]

@Community Alums  But in the System, the Pagination is not present as It comes with Integration Hub enterprise Pack. We have the Integration Hub Professional pack. Any other alternative than pagination.