Data source fields

  • Release version: Washingtondc
  • Updated February 1, 2024
  • 10 minutes to read

    • In addition to common fields, other fields appear on the data source form, depending on the value of the Type field.

    Common data source fields

    These fields are common to all data sources.

    Field Description
    Name A unique name for this data source.
    Import set table label A label for the import set staging table to use.
    Import set table name The ServiceNow AI Platform uses the label you entered to construct a unique table name. This prevents namespace collision with an already existing table.
    Type The type of data you are importing:
    • From a remote File
    • From a JDBC database
    • Using LDAP
    • Using OIDC
    • Using REST to import data through the Integration Hub
    • From a Data Stream action through the Integration Hub
    • Using a Custom script
    Use Batch Import If selected, specifies that loading data should be batched when you click Load data or Test load 20 records.
    Batch Size Specifies how big the batches should be. Appears only when the Use Batch Import field is selected.
    The glide.import_set_load_usebatch and glide.import_set_load_batch_size properties affects these fields. The priority order for the properties and fields are as follows: If the property is set, use the property value. If not, use the field value. If neither property is set, use the default batch size of 1000.
    • For example, if glide.import_set_load_usebatch = true and glide.import_set_load_batch_size = 200, use 200 as the batch size, regardless of the field values.
    • For example, if glide.import_set_load_usebatch = true and the glide.import_set_load_batch_size property is not set, use the default batch size of 1000.
    • For example, if the glide.import_set_load_usebatch property is not set and the Use Batch Import field = true, use the Batch Size field value (or the default of 1000 if the Batch Size field value is not set)
    • For example, if the glide.import_set_load_usebatch property = false and the Use Batch Import field = true, do not use batch import, since the property is set to false.
    Application Application scope for this record.

    File type data source fields

    These additional fields appear on the data source form when the Type field value is File.

    Field Description
    Format Only the default XML-based file format (.xlsx) is supported, unless saved as "Strict Open XML Document."
    Zipped Selected if the import file is compressed.
    File retrieval method The retrieval method for this file.
    File path The path to the file to import.
    You can include a system property value in this field using the format ${system.property.name}. When the connection is made, the variable is replaced with the value of the specified property.
    Note:
    For files retrieved using HTTP or HTTPS, the glide.ds.file.http.variable_replacement property must be true to use system property expansion.

    For example, you can specify the File path value /countries.csv?key=${datasource.apikey}. When the connection is made, the value of the datasource.apikey system property is passed as the value for the key parameter.

    Note:
    For HTTP and HTTPS protocols, the file path is automatically URL encoded. Do not specify a URL-encoded file path when using either of these protocols.
    CSV delimiter The delimiter character for the CSV file.
    Parsing script The script used to parse a single line of input. As you type the script, script auto-complete presents the choices available to you. For more information, see File type data sources.
    Sheet number The Excel sheet number to load data.
    Header row The Excel row number to load the column labels.
    Path for each row The path expression used to select JSON objects, which will become rows in the import set table. The path should be // for JSON objects.
    Discard Arrays Selected to discard JSON arrays as a value. Unselected to serialize JSON arrays as a value.
    Expand node children Selected if child elements of the XML node or JSON object should be converted into additional columns. Unselected if the parent column value should be an XML or JSON fragment. For JSON, note that selecting this option expands objects only, not arrays. For nested array support, see JSON file import examples.
    Data in single column Selected to store data in a single column.
    Xpath for each row XPath expression that selects the nodes which become rows in the import table. The children of the selected nodes will become the columns in the rows.
    SCP authentication method Choice to authenticate with a username and password or with a public key.
    Note:
    You cannot authenticate to data sources with a public-private key pair. Use a username and password authentication instead.
    Server Name of the server from which the tables will be imported.
    Port Port to use to connect to the specified server. This field appears if you select HTTP or HTTPS as the file retrieval method.
    Username User name for authentication on the JDBC server.
    Password Password for authentication on the JDBC server.
    Private keyfile Keyfile when using legacy SCP public-private keyfiles.
    Note:
    • You cannot authenticate to data sources with a public-private key pair. Use username and password authentication instead.
    • Do not use this legacy option. Although it is available for use, it is not supported.
    System keystore Selected to validate the certificate from the FTPS server against all saved certificates. This certificate may be any type supported by the instance. If not selected, the instance uses the Java default certificate to validate the FTPS server. This field is available only for data sources with a File retrieval method value of FTPS. All FTPS file retrieval methods are supported, including Auth TLS, Auth SSL, Implicit SSL, and Implicit TLS.

    JDBC type data source fields

    These additional fields appear on the data source form when the Type field value is JDBC.

    Field Description
    Use MID Server MID Server to use to connect to this data source.
    Format One of the following formats: MySQL, Oracle, or SQLServer.
    Instance name Used to support dynamic port connectivity with SQL Server. To ensure connectivity, provide the instance name for a SQL Server that receives a new port number dynamically after each reboot. This field only appears when SQLServer is selected as a data source format.
    Database name Name of the database instance.
    Database port Port number for the database. Leave this field empty when using dynamic port connectivity with SQL Server.
    Use integrated authentication Selected to allow the JDBC connection to use the ID of the user configured for the Windows MID Server service for SQL Server authentication. For additional details, see JDBC type data source.
    Username User name for authentication on the JDBC server.
    Password Password for authentication on the JDBC server.
    Server Name of the server from which the tables will be imported.
    Query Type of query to run: All rows from Table or Specific SQL. If you select to run a SQL statement, the SQL statement field appears.
    Query timeout Number of seconds the JDBC driver will wait for a query (SELECT) to complete. Zero (0) means no timeout. If timeout is exceeded, the integration considers the JDBC result inaccessible and places it in an error state.
    Connection timeout Number of seconds before MID Server connection cache pool closes and removes it from the pool. Zero (0) means no timeout.
    Table name Name of the table from which the data is being exported.
    SQL statement SQL statement to extract the desired data from the database.
    Use last run datetime Controls the amount of data that is retrieved from a database during an import run. If unselected, then all rows in the table specified are imported, every time. You might want to use this setting if this is a one-time import, or if all the data in the target table is new. If selected, two additional fields appear, enabling you to select a datetime value to limit imported data to delta values only.
    Last run datetime The datetime value in this field is automatically populated from the database field you select below and represents the latest value from the previous run. This value acts as a dynamic filter to restrict the number of records retrieved to those records that have changed since the data source's last runtime.
    Last run database field Field name from the source table that will be used as the filter in the next run. This value may need to be case sensitive, depending on the target database type.
    Additional Database Parameters Optional parameters to add to the end of the URL generated for this data source. Semicolon-separated list of parameters. You may need to configure the form to view this field.

    LDAP type data source field

    This additional field appears on the data source form when the Type field value is LDAP.

    Field Description
    LDAP target Target selected from the list of your LDAP OU definitions.

    OIDC type data source field

    This additional field appears on the data source form when the Type field value is OIDC.

    Field Description
    Format Defaults to -- None -- and can't be changed.

    REST (Integration Hub) type data source fields

    These additional fields appear on the data source form when the Type field value is REST (IntegrationHub).

    Field Description
    Format One of the following formats: Custom (Parse by Script), JSON, or XML.
    Request action Data source request action that will be invoked to get data. You can choose only published request actions.

    Click the link in the dialog box below to create/configure a new Data Source request action in Integration Hub. After the request action is published in Integration Hub, you can choose it as a Request action in the Data Source record. For more information, see Define action properties.
    Support pagination Selected to enable pagination for the request action. Pagination helps you reduce processing overhead by processing a specified number of records at a time, instead of processing all records at once.
    Limit Maximum number of records per page. For example, to process 50 records at a time, enter a Limit of 50.
    Offset The initial offset used for pagination. The offset is the starting record number. For example, to process 50 records at a time, starting with the first record, enter a Limit of 50 and an Offset of 0.
    Parsing script The script used to parse a single line of input. As you type the script, script auto-complete presents the choices available to you. For more information, see File type data sources.
    Path for each row The path expression used to select JSON objects, which will become rows in the import set table. The path should be // for JSON objects.
    Discard Arrays Selected to discard JSON arrays as a value. Unselected to serialize JSON arrays as a value.
    Expand node children Selected if child elements of the XML node or JSON object should be converted into additional columns. Unselected if the parent column value should be an XML or JSON fragment. For JSON, note that selecting this option expands objects only, not arrays. For nested array support, see JSON file import examples.
    Data in single column Selected to store data in a single column.
    Xpath for each row XPath expression that selects the nodes which become rows in the import table. The children of the selected nodes will become the columns in the rows.

    Data Stream (Integration Hub) type data source fields

    These additional fields appear on the data source form when the Type field value is Data Stream (IntegrationHub).

    Field Description
    Type Data Stream (Integration Hub) type data source.
    Data Stream action
    • Select Data Stream action Selected to use an existing published Data Stream action from Integration Hub.
    • Select published Data Stream action or create new Data Stream action Selected for one of the following reasons:
      • If you select a published Data Stream action, click this button to open the Data Stream action in Integration Hub.
      • If you don't select a Data Stream action, click this button to create and publish a new Data Stream action in Integration Hub.
    Note:
    Data Stream actions require a subscription for production use.
    Connection

    Specify the connection alias for the Data Stream action. You can use the default parent alias or select any of its children's aliases. Available child aliases are listed in the menu.

    This field is only available when the Data Stream action is configured to use a connection alias. For more information, see Create a Connection and Credential alias.
    Data in single column Selected to store data in a single column.
    Data Stream action inputs Additional fields may appear, based on the inputs defined in the Data Stream action.

    Custom (Load by Script) type data source field

    This additional field appears on the data source form when the Type field value is Custom (Load by Script).

    Field Description
    Data Loader Script used to load data. For more information, see Custom (Load by Script) type data source