Pagination in Rest API for retrieving records from third party

[aamir1]

Hi All,

I need to retrieve data from a third party and I am able to achieve that but we need to apply pagination to get the data in small numbers. I am not familiar with Pagination concept, could anyone please help me how to achieve this. I am able to achieve the 1st record but getting issues in putting it in a loop.

Here are the codes-

Test: function(){
var now = new Date().toUTCString();
var dt=now.split("GMT");
var date=dt[0]+" -0000";
var method=["GET"];
var path=["/admin/v1/users"];
var param={"limit":1000,"offset":0};
var end="api-****.duosecurity.com";
var sig = this.sign(method, path, param,date);
var req = {
method: method,
headers: {
'Date': sig.date,
'Authorization': sig.authorization
}
};
try {
var r = new sn_ws.RESTMessageV2();
r.setEndpoint("https://api-****.duosecurity.com/admin/v1/users");
r.setRequestHeader('Content_type','application/x-www-form-urlencoded');
r.setRequestHeader('Authorization',req.headers.Authorization);
r.setHttpMethod(req.method);
r.setRequestHeader('Date',req.headers.Date);
var response = r.execute();
var responseBody=response.getBody();
responseBody = JSON.parse(responseBody);
var httpStatus = response.getStatusCode();

if(httpStatus == "200"){
return responseBody;
}
}
}

here is the response that i am getting-

{"metadata": {"next_offset": 1000, "total_objects": 4532}, "response": [................], "stat": "OK"}

[aamir1]

Can you please share an example

[Tony Chatfield1]

Normally I just use a crude loop when initially syncing data, setting the iteration count manually (after a few tests), this is to ensure my authentication isn’t going to fail due to timeout while the loop is running.

Once sync’d I would expect to run a post (or get) frequently enough to ensure everything created\updated during that period is sync’d without need to paginate, but you could easily paginate for higher volumes just by, extracting the result count from your returned payload and terminating once the loop doesn’t return a full set of data.

Try something like this  

var rowCount = 300;
var rowOffSet = 0;

for(var i = 0; i <10; i++) {

 
//your code with this line updated to reference the variables.

var param={"limit": rowCount,"offset": rowOffSet};

 

rowOffSet = rowOffSet + rowCount;

}

[aamir1]

Thanks Tony for your response. However, It didn't work this way. 

[Tony Chatfield1]

What exactly didn't work and what have you done to debug/resolve the issue?