How to bring data into ServiceNow through REST Step with Flow

Chris Roser · ‎04-18-2022

I'm currently working on bringing VM information into the CMDB. Currently I have a REST Step Action setup within Flow Designer that is currently bringing info in. However, the data is paginated. Because I don't have Data Streams (IntegrationHub Enterprise) I have to have a separate Flow that runs a script to cycle through the pages and pull the VM information 200 items at a time.
Where I'm stuck is I need to know how in the world to get this information into the CMDB from here. Initially I was looking into IntegrationHub ETL, but that fundtions off of a Flow Action, not an entire Flow. Basically need to know how to get past the pagination hurdle and get the information imported into CMDB.
Whether through what I've done so far, or some other way I'm not aware of, I'm open to any and all information I just haven't been able to find much to get this setup. Thank you!

Report Inappropriate Content · ‎04-20-2022

If you can download the Service Graph Connector for SolarWinds in a development instance it would be a good example of how to do this.

It goes from a scheduled data import -> data source -> IntegrationHub Rest Action -> import set table -> RTE -> IRE -> CMDB

You would configure when you want it to run via a scheduled import. I have attached some screenshots to help you, but as mentioned actually seeing it in an instance would be more beneficial.

I had to remove a lot of script to get this in one screenshot, but it captures the intent:

In SolarWinds case, it's fairly basic so we don't actually need a scripted data source. The API is a JSON POST that takes a sql like query. The pagination has to be part of that query so we modify the query in script before sending the REST request.

View solution in original post

Report Inappropriate Content · ‎04-18-2022

I think you are on the right track, just slightly off. I would look into creating a custom data source. From there you can call your flow (or directly call the rest action and remove the flow as you should be able to handle the flow logic in the script) and populate an import set table with that data. Once you have the data in the import set table you can turn it over to the IntegrationHub ETL to transform and map the data into the CMDB.

Chris Roser · ‎04-19-2022

"or directly call the rest action and remove the flow as you should be able to handle the flow logic in the script"
What do you mean by this? That I can somehow have the pagination handling work within the Action somehow via a script step?
Currently in the Flow I have flow variables that have to be setup in the flow so that I can call / store on them in the Flow. Is there some way to do that on the Action side? Or am I misunderstanding?

Report Inappropriate Content · ‎04-19-2022

I don't know if I can definitively answer that. Paginated APIs aren't standardized, but they general require some parameters such as a page size and current page. You can have those pagination variables as inputs to your action and then pass them in from script. The script can have knowledge of what page is currently being fetched and increment to the next page in a loop. Then you can break when the response is empty.

Chris Roser · ‎04-19-2022

I see what you're getting at, thank you for the explanation. This is basically what I'm currently doing in the Flow, I just don't get how it'd be handled in the Action itself. You're basically stating it would have to be custom scripted out as opposed to using the Flow and it's 'low code' loop? Also part of the reason we had it associated into a Flow is this is going to be a repeat import that runs daily to update any VMs that have been spun up / retired. Is there a way to do any of this with automation included?