How to use the DataStream action and Microsoft Graph API rest with Pagination.

[Community Alums]

Hello Community,

I am trying to use the DataStream action in FlowDesigner to retrieve information from Microsoft Graph API.   I am able to pull the 20 record limit when testing my action in flow designer but I am trying to enable pagination as I know there are multiple pages of records that will be returned.   I am just looking to see if anyone else has been able to create a datastream and leverage the @odata.nextLink values for pagination.

I would appreciate anyones insights.

[Pascal Verdieu]

You need 2 things to make this work:

1. In the Pagination Setup Step, you need to create a script which will extract the skipToken off the response
   
2. In the REST step, you need to  use the skipToken from the previous step
   

For reference, here's the script I'm using:

(function paginate(variables, pageResponse) {
    // Use this script to set pagination variables or 
    // manipulate variables extracted by XPath or JSON Path
    try {
        gs.info('Next Link: ' + variables.nextLink);
        if (variables.nextLink != undefined && variables.nextLink != "") {
            var regex = /\$skipToken=(\w+)/i;
            variables.skipToken = regex.exec(variables.nextLink)[1];
            gs.info('Skip Token: ' + variables.skipToken);
            variables.getNextPage = variables.skipToken != undefined && variables.skipToken != "";
        } else {
            gs.info('Empty Next Link');
            variables.getNextPage = false;
        }
    } catch (e) {
        gs.error("Pagination error: " + e.message);
        variables.getNextPage = false;
    }
})(variables, pageResponse);

HTH

[Shreya Shah]

When you test a data stream action it limits to 20 records because its in "test" mode, however when you run the actual action within a flow, you will be able to get all the records. Please create a flow and try to use your action within it to get more results.

other way is to open any other data stream action from the Microsoft Teams Graph Spoke and see its implementation and copy the action and modify to your use case.

[Pascal Verdieu]

Good point, I forgot about that.  To test pagination, what I did was to add in the REST Step a small (like 2 or 5) $top query parameter, then when I had pagination right, I'd remove it.