Custom (Load by Script) type data source

  • Release version: Zurich
  • Updated July 31, 2025
  • 3 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Custom (Load by Script) type data source

    The Custom (Load by Script) data source type in ServiceNow allows you to use JavaScript to fetch and insert any type of incoming data into an import set table. This method is ideal for integrating data from various sources such as REST APIs or file attachments. The key feature is theData Loaderscript, where you write custom code to load data into the import set table.

    Show full answer Show less

    Key Features

    • Data Loader Script: A JavaScript field where you define how to fetch and insert data. It uses parameters like importsettable to insert data, and supports methods to add columns and insert rows.
    • Parallel Loading: Enables partitioning of data into smaller chunks to load in parallel, improving import speed and reducing impact on other processes. Parallel loading requires the import to be concurrent and the partition method set to Custom size.
    • Input Parameters for Scripts: The Data Loader script receives parameters such as importsettable, datasource, importlog, lastsuccessimporttime, and partitioninfo (for parallel loading).
    • Parallel Loading Script: Uses parameters like paralleljobloader to manage partitions and schedule parallel jobs, including delayed jobs when data is not immediately available.
    • Job Management and Monitoring: You can monitor parallel jobs via the Concurrent Import Sets table and Events table. Delayed jobs show as Pending. Jobs stuck in Running for over 600 minutes revert to Pending automatically, with configurable retry limits and auto-retry settings.
    • Script Auto-Complete: The scripting environment offers auto-complete to assist in writing Data Loader scripts efficiently.

    Key Outcomes

    • Allows flexible and powerful integration of data from diverse sources using custom scripting.
    • Improves data import performance with parallel loading, reducing overall integration time.
    • Provides mechanisms to manage, delay, retry, and monitor parallel import jobs to ensure reliability and control.
    • Supports custom data structures by letting you dynamically add columns of different types (string, JSON, XML) to the import set table.

    Use a custom script to store any type of incoming data in the import set table.

    When you select a Type of Custom (Load by Script) for your data source, the Data Loader field appears. The Data Loader field is a JavaScript field where you can enter a script to fetch data and insert it into the import set table using the import_set_table input parameter. For example, you can enter a script to get data from a REST API or a file attachment.

    The Data Loader script loads the data. You can also Enable parallel loading to use a script to partition the data into smaller sections, then load them in parallel. Parallel loading can enable your integrations to finish in less time and create less of an impact on other tasks. Parallel jobs are usually processed immediately. If you need to delay a job, you can configure a delay in the Data Loader script.
    Note:
    With parallel loading, you must be running a concurrent import and the Partition Method for the import must be set to Custom size. For more information see, Schedule a data import.

    Data Loader script

    The data loader script has the following input parameters.
    • import_set_table: The import set table referred to in the Data Source record. The data is inserted into this table.
    • data_source: The data source referred to in the Data Source record.
    • import_log: The log that records information about data import activity.
    • last_success_import_time: The last time this data source was run successfully.
    • partition_info: The partitioning information for the data. Used for parallel loading.
    The import_set_table parameter has the following methods:
    • addColumn(label, maxLength): Adds a string-type column to the import set table.
    • addJSONColumn(label, maxLength): Adds a JSON-type column to the import set table.
    • addXMLColumn(label, maxLength): Adds an XML-type column to the import set table.
    • insert(rowData): Inserts a map (key = column name, value = column value) in the import set table.
    • getMaximumRows(): Returns 20 when you select Test load 20 records. In every other case, returns -1.

    Data Loader example

    Data Loader example showing a script that inserts data into the import set table.

    Parallel loading script

    The parallel loading script has the following input parameters.
    • parallel_job_loader: The Parallel Jobs [sys_parallel_job] table.
    • data_source: The data source referred to in the Data Source record.
    • import_log: The log that records information about data import activity.
    • last_success_import_time: The last time this data source was run successfully.

    The parallel_job_loader parameter has the add(partitionInfo) method, which adds the partition information to the Parallel Jobs [sys_parallel_job] table.

    Parallel loading example

    In this example, the parallel loading script partitions the data into smaller sections to be loaded in parallel. The Data Loader script loads the data into the import set table and adds a new partition to process data after a 120-second delay.

    Parallel loading script example.

    Parallel loading delays, status, and properties

    For most parallel jobs, data is available when the Data Loader script runs, so you don’t need to add any parallel jobs to the Data Loader script.

    In cases where the data might not be immediately available, you can create an additional parallel job with a delay in the Data Loader script. When the Data Loader script runs, the parallel job is scheduled to run after the amount of time specified by the delay.

    When adding parallel jobs to the Data Loader script, avoid repetitive loops. You can avoid repetitive loops by specifying a constraint. In the previous example, the parallel job in the Data Loader script uses the constraint  (partition.start < 1000). Delays are measured in seconds.

    You can check the status of your current parallel import tasks by going to the Concurrent Import Sets [sys_concurrent_import_set] table, selecting your job, and viewing the Parallel Jobs tab.

    Delayed jobs may show as Pending depending on the delay’s length of time. Parallel jobs are processed from the event queue, so you can also view your jobs by going to the Events [sysevent] table and filtering by [Queue] [is] [impex_parallel_job_queue].

    If a job gets stuck in a Running state for more than 600 minutes, it's automatically put back into a Pending state. You can modify the amount of time a parallel job is allowed to be stuck in a running state with the com.glide.system_parallel_job_inactive_time_minutes property.

    If a parallel job can't complete after two tries, the job is set to Error. You can modify the number of tries before having an error state with the com.glide.system_parallel_job_max_retry_count property. The com.glide.system_parallel_job_auto_retry property can be set to false to disable auto retry.

    Script auto-complete

    As you type the script, the script auto-complete displays the available options, as shown in this example.

    Data Loader auto-complete