Data Stream action design considerations

  • Release version: Zurich
  • Updated July 31, 2025
  • 4 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Data Stream action design considerations

    This guide outlines best practices and important considerations for ServiceNow customers when creating and integrating Data Stream actions into flows. Following these guidelines helps prevent errors, performance bottlenecks, and ensures reliable operation of Data Stream actions, which require a constant connection to the response stream.

    Show full answer Show less

    Considerations when creating a Data Stream action

    • Pagination Logic: Develop pagination scripts based on third-party API requirements. Use the getNextPage Boolean variable to control continuation, setting it to true if a next page token exists, otherwise false. Access variables via bracket or dot notation.
    • Preprocessing Scripts: Avoid scripts that pause the action or call a MID Server, as Data Stream actions need uninterrupted streaming. Any pause causes an error.
    • Data Type Conversion: Pagination variables are strings; convert them to integers for math operations, then convert back to strings to maintain compatibility.
    • Pagination End Condition: Ensure your pagination script has a clear stop condition to prevent infinite loops and excessive flow runtimes.
    • Testing: Thoroughly test Data Stream actions before deploying to production to catch errors early.
    • Configuration Page Management: Clearing options removes associated steps and their data—handle this carefully to avoid accidental data loss.
    • Page Size: Keep each returned page under 1 GB to avoid MID Server size limit issues. Larger sizes may require adjusting attachment size limits.
    • Nesting: Nesting Data Stream actions is not supported and should be avoided.
    • Retry Policy: Only fixed interval retry strategies are supported, with a maximum total delay of 30 seconds (interval multiplied by retry count). Data Stream steps do not enter a waiting state during retries.

    Considerations when adding a Data Stream action to a flow

    • No Wait Conditions in For Each Loops: Avoid adding actions that pause flows, such as "Ask for Approval" or "Wait for Condition," inside Data Stream For Each logic blocks. These actions interrupt the needed continuous stream connection.
    • MID Server Execution Restrictions: Do not add actions that run on a MID Server (e.g., JDBC, PowerShell, SSH steps) within Data Stream For Each blocks, as these pause or disrupt streaming. Actions that may run on a MID Server depending on configuration (REST, Script, SOAP steps) should be verified to ensure they do not run on the MID Server at runtime.
    • Performance Optimization: When processing large record sets in For Each blocks, disable reporting by setting the com.snc.processflow.reporting.level system property to reduce resource consumption and improve flow performance.

    Follow these considerations when creating a Data Stream action and when adding a Data Stream action to a flow to prevent errors and performance issues.

    Considerations when creating a Data Stream action

    Create Data Stream actions with these considerations in mind.

    Write pagination logic according to third-party requirements

    Evaluate and understand the format required by your third-party endpoint. For example, you may need to write a script that sets the built-in getNextPage variable to true as long as there is a nextPage token in the response. If the response does not contain the token, then set the variable to false. Access variables in script using bracket or dot notation. For example, variables['getNextPage']. This variable only accepts the Boolean data type. The default value is false.

    Ensure that preprocessing scripts do not call a MID Server or pause the action.
    Data Stream actions require a constant connection to the response stream. They do not support preprocessing scripts that pause the action to call a MID Server or pause the action to wait for a condition or duration. A preprocessing script that pauses the action for any reason returns an error message.
    Convert pagination variable data types to perform math operations

    Pagination variables only support the string data type. To perform math operations, you must convert the value to an integer, perform any required operations, then convert it back to a string.

    variables['offset'] = (parseInt(variables['offset']) + parseInt(variables['limit'])).toString();
    Ensure that the pagination script has an end condition
    Avoid infinite loops in pagination requests by creating a condition that sets the getNextPage variable to false. Cancel any long-running flows. Always test Data Stream actions before using them in production.
    Clear configuration page options carefully
    Clearing an option in a configuration page removes the step from the Data Stream outline and deletes all data associated with the step.
    Consider the size of each page returned
    When configuring pagination options, try to ensure that the size of each page returned is under one GB. This ensures that flow designers who use the action won't encounter size limit issues when retrieving data from a MID Server. If the action returns more than one GB of data per page, the flow designer may need to increase the attachment size limit. For more information, see MID Server support for Data Stream actions.
    Note:
    • Nesting Data Stream actions is not supported.
    • When using the retry policy, ensure that you are aware of these considerations:
      • Only a fixed time interval strategy is honoured for the data stream actions.
      • The total delay (count multiplied by the interval) can't exceed 30 seconds. For example, if you want retry for a maximum of 3 times, the maximum delay for each retry is 10 seconds.
      • Data stream step doesn’t go into the waiting state during retry.

    Considerations when adding a Data Stream action to a flow

    Add a Data Stream action to a flow with these considerations in mind.

    Do not add actions with wait conditions to a Data Stream For Each logic block

    You cannot add an action that pauses the flow to a Data Stream For Each flow logic block. Data Stream actions require a constant connection to the response stream. For example, you cannot use the Ask for Approval or Wait for Condition actions within Data Stream For Each flow logic.

    Some actions and steps always pause a flow and cannot be added to Data Stream For Each flow logic blocks. However, some actions and steps may pause the flow depending on how they are configured. Check the action configuration to ensure it does not pause the flow at runtime.

    Always pauses the flow (unsupported with Data Stream actions) Ask for Approval action
    Ask for Approval step
    Wait for Condition action
    Wait for Condition step
    May pause the flow depending on configuration Create Catalog Task action
    Create Task action
    Create Task step
    Do not add actions that run on the MID Server to a Data Stream For Each logic block

    Some actions and steps always run on a MID Server and cannot be added to Data Stream For Each flow logic blocks. However, some actions and steps may run on a MID Server depending on how they are configured. Check the action configuration to ensure it does not run on the MID Server at runtime.

    Always runs on a MID Server (unsupported with Data Stream actions) JDBC step
    PowerShell step
    SSH step
    May run on a MID Server depending on configuration REST step
    Script step
    SOAP step
    Turn off reporting when consuming a large number of records in the For Each logic block
    Iterating over a large number of records can be resource intensive, especially when the For Each logic block includes complex actions for each iteration. To avoid performance issues, turn off reporting using the com.snc.process_flow.reporting.level system property. For more information, see Workflow Studio flow system properties.