---
sourceDocument: Zurich Workflow Data Fabric
sourceDocumentLink: https://www.servicenow.com/docs/r/zurich/integrate-applications

 Release :

    - zurich

ft:locale :

    - en-US

ft:publication_title :

    - Zurich Workflow Data Fabric

ft:clusterId :

    - crint

bundleId :

    - crint

workflow :

    - Creator


---

# Data Stream actions and pagination

# Data Stream actions and pagination {#ariaid-title1}

* Release version: Zurich
* 
* Updated July 31, 2025
* 
* ![](https://www.servicenow.com/docs/portal-asset/ico-clock) 10 minutes to read

Summarize  
![AI sparkle icon](https://servicenow.com/docs/portal-asset/ai-sparkle-icon) Summarized using AI  
This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.  

## Summary of Data Stream actions and pagination

Data Stream actions in ServiceNow's Workflow Studio enable you to send REST, SOAP, or JDBC requests to APIs that return large streams of data (over 10 MB) or paginated results.
These actions parse the response into complex object outputs, which you can use in flows for further processing, such as importing large datasets like employee records into the User \[sysuser\] table.
Note that using Data Stream actions requires an Integration Hub subscription.
Show full answer Show less  

## Key Features

* **Handling Large and Paginated Data:** Parse and format large response data and automatically manage multiple paginated API requests.
* **Integration Hub Use:** Can be utilized within Integration Hub Import and to create data sources, allowing reuse across multiple flows.
* **For Each Flow Logic:** Data Stream actions automatically integrate with For each flow logic blocks, enabling processing of each individual object in the data stream, facilitating record creation or other operations for each item.
* **Dual Execution Methods:** Run Data Stream actions either from a flow or programmatically via the executeDataStreamAction() method in the FlowAPI class.
* **Structured Action Outline:** Actions follow a predefined step structure with categories such as Action Preprocessing, Request, and Parsing, supporting REST, SOAP, and JDBC APIs.
* **Pagination Setup:** Configure pagination with templates (e.g., Limit/Offset) for REST and SOAP APIs to manage multiple pages of results; JDBC does not support pagination.
* **Parsing Capabilities:** Use Splitter steps (JSONPath or XPath) to identify repeated elements and Script Parser steps to map data into complex objects for REST and SOAP responses.
* **JDBC Specifics:** JDBC Data Stream actions retrieve entire datasets in a single request, do not support pagination, and allow optional transform scripts for data formatting.
* **Error Evaluation:** Define error handling behavior and custom error conditions at each step to manage failures robustly.
* **Execution Details and Reporting:** Access detailed runtime and configuration data for processed items and pages to monitor Data Stream action performance and troubleshoot issues.
* **MID Server Support and Properties:** JDBC Data Stream actions require MID Server execution with configurable timeout and payload size properties to optimize data retrieval.

## Practical Application for ServiceNow Customers

Data Stream actions empower ServiceNow flow designers to efficiently import and process large volumes of data from external systems with minimal coding. By automating pagination and parsing, they simplify integrations with third-party APIs, making it easier to maintain up-to-date records such as users or documents. The built-in error handling and execution monitoring features help ensure reliable data processing. Additionally, reusable Data Stream actions enhance consistency and reduce duplication across multiple flows.

## Design and Usage Considerations

* Ensure Integration Hub subscription is active to use Data Stream actions.
* Configure pagination appropriately for REST and SOAP APIs to avoid missing data.
* Use the automatic parsing generation feature for REST-based Data Stream actions to accelerate setup.
* Leverage For each flow logic integration to handle individual data objects within flows effectively.
* For JDBC data streams, adjust MID Server timeout and payload size properties to suit data retrieval needs.
* Run tests on Data Stream actions before deploying them in flows to validate functionality.  
Send REST, SOAP, or JDBC requests from Workflow Studio to APIs that return a stream of response data larger than 10 MB, or that return paginated results. Parse stream data into a series of complex object outputs and use the data pills in other actions in a flow.  
For example, create a Data Stream action to import a large quantity of employee data from a third-party HR site. The Data Stream action sends a REST request to the third-party site and processes the response to populate records in the User \[sys_user\] table.  
Note:  
Data Stream actions require an Integration Hub subscription. For more information, see [Legal schedules - Integration Hub overview](https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/legal/snc-addendum-integrationhub.pdf).

## Benefits {#data-stream-actions__section_d3h_2pm_tgb}

Data Stream actions offer these benefits.  
* Parse and format a stream of response data larger than 10 MB.
* Automatically send multiple requests to APIs that paginate results, if applicable.
* Can be used in [Integration Hub - Import](https://www.servicenow.com/docs/sx1DOvdAo99ogo22YnNUhQ "Automate data import tasks through a consolidated interface that provides a guided, step-by-step experience. Define external data sources, specify target tables, map how source data becomes target data, and schedule regular imports.") and to [create a data source](https://www.servicenow.com/docs/b_evTXCJBPugxyf~DQyMnA "Create a Data Stream (IntegrationHub) type data source record to define what complex data an import set should import from a Data Stream action.").
* Enable flow designers to process large requests without complex coding or configuration.
* Enable flow designers to process each object within a data stream using For each flow logic. For example, you might create a Data Stream action that imports document data from a third-party site. When you add the action to a flow, Workflow Studio automatically adds the action to a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream. See [Use a Data Stream action in a flow](https://www.servicenow.com/docs/gqi~enjPTAE6Zxso444ufQ "Data Stream actions use the For each flow logic option to process stream data in a flow. For example, suppose that you want to use a Data Stream action that imports document data from a third-party site. When added to a flow, the action appears as a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream.").
* Enable flow designers to reuse Data Stream actions in multiple flows, using the same source of data in multiple ways.
{#data-stream-actions__ul_k2p_2pm_tgb}

## Running a Data Stream action {#data-stream-actions__section_dq2_x13_z3b}

There are two ways to run a Data Stream action.  

From a flow
:   You can process each object within a data stream using For each flow logic. For example, you might create a Data Stream action that imports document data from a third-party site. When you add the action to a flow, Workflow Studio automatically adds the action to a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream. See [Use a Data Stream action in a flow](https://www.servicenow.com/docs/gqi~enjPTAE6Zxso444ufQ "Data Stream actions use the For each flow logic option to process stream data in a flow. For example, suppose that you want to use a Data Stream action that imports document data from a third-party site. When added to a flow, the action appears as a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream.").

From a script
:   You can start a Data Stream using the executeDataStreamAction() method in the FlowAPI class. For more information, see [FlowAPI](https://www.servicenow.com/docs/access?context=ScriptableFlowAPI&version=zurich&pubname=zurich-api-reference&ft:locale=en-US).

## Action outline {#data-stream-actions__section_wnp_sgg_5gb}

Data Stream actions follow a set structure. Follow prompts to add and remove steps from the action outline. You cannot manually add steps to a Data Stream action.
Figure 1. REST and SOAP Data Stream actions Figure 2. JDBC Data Stream action  
Note:  
Clearing an option in a configuration page removes the step from the Data Stream outline and deletes all data associated with the step.

## Action error evaluation {#data-stream-actions__section_ewd_xdp_qdc}

Use [error evaluation](https://www.servicenow.com/docs/access?context=action-error-evaluation&version=zurich&pubname=zurich-build-workflows&ft:locale=en-US) to catch step errors and specify the error behavior of each step you add to a data stream action. You can also create your own error conditions by
specifying when an action returns an error state as well as the status codes and messages they return.

## Action Preprocessing {#data-stream-actions__section_c3c_l1t_tgb}

Use the Action Preprocessing category to retrieve connection and credential details or to run a preprocessing script.

Select Retrieve connection info to retrieve connection and credential details to use in your action. Selecting this option adds the Get Connection Info step as the first step in the action preprocessing.
For more information, see [Get Connection Info step](https://www.servicenow.com/docs/access?context=get-connection-info-action-designer&version=zurich&pubname=zurich-build-workflows&ft:locale=en-US).

Select Enable preprocessing script to run a preprocessing script before the action sends the initial API request. For example, validate action inputs or set default values. Selecting this option adds a script step to the Data Stream action. For more information, see [Script step](https://www.servicenow.com/docs/access?context=javascript-step-action-designer&version=zurich&pubname=zurich-build-workflows&ft:locale=en-US).

Preprocessing executes once per action, before the first API request.

This is an optional Data Stream action component that runs on either the instance or a MID Server.

## Request {#data-stream-actions__section_a32_m1t_tgb}

Use the Request category to configure how the action sends API requests. The Request
section executes once per page of results. Request components provide these configuration
options.  

Pagination Setup step

:   Request results in batches. Once one page of data is processed, the Data Stream action runs the request section again to return the next set of results. Use the pagination setup step to set up pagination options required by the API. Configure the Pagination Setup step manually, or select a pre-built template
    to apply common configurations. For example, apply the Limit / Offset template to specify the number of items you want returned per page (limit), and the starting number for the first item (offset). After applying a template, update the values to ensure that the configuration complies with the API's requirements.

    Note:  
    For licensing purposes, each request counts as one transaction, including each request for the next page of results.

    The value of the reserved, read-only `getNextPage` variable determines whether to request another page of results. As long as the `getNextPage` variable is true and the previous page contains data, the action continues to send requests for the next page.  
    Note:  
    You must explicitly set the value to true in the script or it will default to false.  
    This is an optional Request component that only runs on the instance.  
    Note:  
    Pagination isn't applicable to the JDBC step.

Script step

:   Run a script before every request for the next page of results. Use this script for data validation and transformation when calling a paginated API. For example, generate a JSON payload for the next page request. Selecting this option adds a script step to the Data Stream action. For more information, see [Script step](https://www.servicenow.com/docs/access?context=javascript-step-action-designer&version=zurich&pubname=zurich-build-workflows&ft:locale=en-US).

    This is an optional Request component that runs on either the instance or a MID Server.

REST or SOAP step

:   Send a REST or SOAP request to a third-party API. Select a data format to add an associated step to the Data Stream action. For more information, see [REST step](https://www.servicenow.com/docs/dBW1kdOzT66JVqYcu1ud0Q "Send an outbound REST web service request to an external system.") and [SOAP step](https://www.servicenow.com/docs/2hDNQIouaXF8P86tG9s7rA "Enable action designers to send outbound SOAP web service requests to external systems.").

    This is a mandatory Request component that runs on either the instance or a MID Server.

JDBC step

:   Send a JDBC request to a third-party API. Use transform script to format data and add an associated step to the Data Stream action. For more information, see [JDBC step](https://www.servicenow.com/docs/6rTG~CZhMgIZGsbTNfdUUA "Create a reusable action to send SQL commands to a relational database.") and [Test JDBC step](https://www.servicenow.com/docs/4XlFrFPpa_UVL7xo1f_85g "Test the JDBC step before testing or publishing an action that contains the JDBC step."). All data is retrieved and pagination is not needed.

    This is a mandatory Request component that runs on a MID Server.

## Parsing in REST and SOAP steps {#data-stream-actions__section_pkp_wws_tgb}

Use the Parsing category to configure how the action separates data stream elements into complex data objects. Use the Splitter step to identify and separate items from an XML or JSON stream, and use the Script Parser step to transform each item into a complex object. The Parsing section executes once per item in the stream. You can access outputs from previous steps in your data stream action using the fd_data object, excluding:

* REST or SOAP step Response Body, Stream, or Error Message outputs
* Splitter step outputs
{#data-stream-actions__ul_e3d_bdb_plb}

<br />

For more information about complex data, see [Complex data](https://www.servicenow.com/docs/access?context=complex-data&version=zurich&pubname=zurich-build-workflows&ft:locale=en-US). Parsing components provide these configuration options.  

Splitter step

:   Identify the parent node in the response stream to map to a complex object. For example, identify a user element in an XML payload to create a complex object for each user in the response stream.

    Select a splitter type to identify and separate repeated items in an XML or JSON data stream.  
    * JSON: Identifies objects from a stream of JSON data. Use a JSONPath expression to identify a JSON array containing repeated data.
    * XML: Identify objects from a stream of XML data. Use an XPath expression to identify an XML element containing repeated data.
    {#data-stream-actions__ul_egw_4p2_fhb}

    This is a mandatory Parsing component that only runs on the instance.

Script Parser step

:   Use JavaScript and ServiceNow APIs to map items in the response stream to a complex object output represented by the `targetObject` global object. For example, map incident record elements identified in the splitter step to a complex object containing incident fields. If the data stream includes siblings to the item identified in the splitter step that you do not want mapped to a complex object, include conditions to exclude those items. You can skip items in the stream by adding `outputs.state = 'SKIP'` wherever needed to the script section of the Script Parser step.

This is a mandatory Parsing component that only runs on the instance.  
Figure 3. REST and SOAP Data Stream action overview

## Generate the parsing phase for REST-based Data Stream actions {#data-stream-actions__section_hlw_y1v_pfc}

You can automatically configure the splitter step, script parser step, and outputs for REST-based Data Stream actions. The Test REST step functionality in REST-based Data Stream actions executes a request to the configured REST
endpoint, analyzes the response payload, and automatically sets up the parsing and output components.

When a REST step is added to the Request section of a Data Stream action, you can use the REST step's Test REST Step button to [auto-generate the Parsing section and Outputs](https://www.servicenow.com/docs/tELYwkKvWU4uIM2aGgIMnQ "Automatically configure the splitter step, script parser step, and outputs for REST-based Data Stream actions. The Test REST step functionality in REST-based Data Stream actions executes a request to the configured REST endpoint, analyzes the response payload, and automatically sets up the parsing and output components."). The Parsing section includes the splitter step and parser step. Auto-generating also puts complex object output in the Outputs
section.

## Transform script in JDBC step {#data-stream-actions__section_sqs_rgj_h4b}

JDBC data stream action doesn't require pagination. Also, splitter and parser steps aren't
required.  
The JDBC step generates a complex object for each of the retrieved record. Hence, action preprocessing and transform script in the JDBC data stream action are optional. When using the transform script, action designer must specify the internal name of the table columns in the transform script.Figure 4. JDBC Data Stream action overview

## JDBC operations and MID Server timeouts {#data-stream-actions__section_gct_cym_mjc}

For JDBC operations, execute the Data Stream Action asynchronously and poll the Attachments \[sys_attachment\] table for results.

The MID Server processes the SQL statement, while the instance/main thread waits for context payloads to be inserted into the attachment table to query the next record.

You can adjust timeout values for JDBC operations with the following properties.  

com.snc.process_flow.datastream.payload.timeout.seconds
:   The amount of time the instance waits for the payload to be available from the JDBC execution in the MID Server. A bounded property with a minimum value of 0 seconds and a maximum value of 7200 seconds. The default
    time is 600 seconds.

com.snc.process_flow.datastream.async_child.timeout.seconds
:   The amount of time allocated for the execution of a child plan in the MID Server.A bounded property with a minimum value of 0 seconds and a maximum value of 7200 seconds. The default time is 60 seconds.

## Data Stream outputs in SOAP and REST steps {#data-stream-actions__section_j4k_rpz_c3b}

When designing a Data Stream action, you must create a single output of type Object or Dynamic Object. The Script Parser step maps items in the stream to this object using the
`targetObject` global object.

At runtime, the system splits and parses the stream of response data according to the Data Stream configuration. Each item in the stream maps to the complex object structure defined by the Script Parser step and the object output, resulting in a large series of
complex objects. For more information about complex data, see [Complex data](https://www.servicenow.com/docs/access?context=complex-data&version=zurich&pubname=zurich-build-workflows&ft:locale=en-US).

## Data Stream outputs in JDBC step {#data-stream-actions__section_j23_glk_h4b}

The output of JDBC steps is a complex object stream. Entire data is retrieved in one request only.  
Note:  
* You can only retrieve data and can't update or delete records using the JDBC data stream action.
* The fields, Maximum Row and Maximum Payload Size (KB) that are available in JDBC step aren't available in the JDBC data stream action.
{#data-stream-actions__ul_a3y_1mk_h4b}

## Execution details in REST and SOAP steps {#data-stream-actions__section_if2_rs2_1hb}

View the configuration and runtime results for each item processed by a Data Stream action. Select a record number to see its configuration and runtime details. By default, the execution details include requests for the last 1000 items. To change the
number of items in the execution details, update the com.snc.process_flow.reporting.datastream.item.lastn system property.

Data stream summary

:   View an overview of the execution that includes this information.

    * Page count: Number of pages returned by a paginated API.
    * Total item count: Number of items in the response stream mapped to complex object outputs.
    * Error count: Number of errors encountered.
    {#data-stream-actions__ul_wbb_scb_nhb}

Page details

:   View runtime data for each step within the Data Stream action. Select a page to view runtime details for each request to a paginated API. By default, the execution details include requests for the last five
    pages. To change the number of requests in the execution details, update the com.snc.process_flow.page.reporting.lastn system property. Set the value to <kbd class="ph userinput">0</kbd> to remove pages
    from the execution details and <kbd class="ph userinput">-1</kbd> to include all pages.

    Note:  
    Including all pages can affect performance and is not recommended.

## Execution details in JDBC step {#data-stream-actions__section_edv_3nk_h4b}

Construction of the output complex object schema isn't needed for the JDBC data stream
action. You can test the query and see the query result. See [Test JDBC step](https://www.servicenow.com/docs/4XlFrFPpa_UVL7xo1f_85g "Test the JDBC step before testing or publishing an action that contains the JDBC step.") for more information. Configure the
MID Server properties
mid.jdbc.datastream.max.record.size and
mid.jdbc.datastream.fail.when.attachement.limit.exceeded to retrieve
data as per your requirement. See [MID Server properties](https://www.servicenow.com/docs/access?context=r_MIDServerProperties&version=zurich&pubname=zurich-servicenow-platform&ft:locale=en-US) for more
information.
* **[Data Stream action design considerations](https://www.servicenow.com/docs/NIiUx5uhtUpfLXhUFcAKnA)**   
  Follow these considerations when creating a Data Stream action and when adding a Data Stream action to a flow to prevent errors and performance issues.
* **[MID Server support for Data Stream actions](https://www.servicenow.com/docs/AFPQqrcFjaRWbI_BatfuHw)**   
  Get data through a ServiceNow® MID Server when running a Data Stream action.
* **[Create a Data Stream action](https://www.servicenow.com/docs/s2GLx5e9l8m23xQ9kXoT3Q)**   
  Create a reusable action to process a stream of response data within a flow.
* **[Generate the Parsing phase for REST-based Data Stream actions](https://www.servicenow.com/docs/tELYwkKvWU4uIM2aGgIMnQ)**   
  Automatically configure the splitter step, script parser step, and outputs for REST-based Data Stream actions. The Test REST step functionality in REST-based Data Stream actions executes a request to the configured REST endpoint, analyzes the response payload, and automatically sets up the parsing and output components.
* **[Test a Data Stream action](https://www.servicenow.com/docs/KMEqroBn6u_ZwJ51NK_d3A)**   
  Test your Data Stream action to ensure that it works the way you expect before you add it to a flow.
* **[Use a Data Stream action in a flow](https://www.servicenow.com/docs/gqi~enjPTAE6Zxso444ufQ)**   
  Data Stream actions use the For each flow logic option to process stream data in a flow. For example, suppose that you want to use a Data Stream action that imports document data from a third-party site. When added to a flow, the action appears as a For each flow logic block, enabling flow designers to easily create a record in ServiceNow for each object in the data stream.