Create Extract Transform Load (ETL) definitions

Create ETL definitions to extract data from a source table, transform the data as desired, and load the data into multiple target tables. ETL definitions also support nested data structures.

Before you begin

Role required: import_transformer

Procedure

1. Navigate to All > System Import Sets > Administration > ETL Definitions.
2. Click New.
3. Complete the form.

   Field	Description
   Name	Name of the ETL definition.
   Description	Description of the ETL definition.
   Copy empty fields	Option to copy empty field values. If the incoming data has a field with an empty value, you can either copy the empty value or ignore it and keep the existing value.
   Execute onBefore script	Option to run a script before inserting or updating data. The onBefore script has three parameters. source: The source data from the import record. target: The transformed data from the import record. In the onBefore script, this parameter accesses transformed data before it's inserted into the target table. importLog: The log that records information about data import activity. The onBefore script has two variables. ignore. Type: Boolean. When true, the specified import set row is ignored. ignore_reason. Type: String. Message describing why the import set row was ignored.
   Execute onAfter script	Option to run a script after inserting or updating data. The onAfter script has three parameters. source: The source data from the import record. target: The transformed data from the import record. In the onAfter script, this parameter accesses transformed data after it's inserted into the target table. Once a record is inserted or updated, it has a sys_id. importLog: The log that records information about data import activity.
   Application	Application scope for this record.
   Active	Selected if the ETL definition is active. Unselected if the ETL definition isn't active.

4. Click Submit.

Example

In this example, the ETL Definition for Worker has both an onBefore and onAfter script. In the onBefore script, the source record with an id of 124 is ignored because that worker has already resigned. The source data and target data are both written to the import log. Finally, the script iterates through the transformed values of the sn_etl_demo_worker table to get each worker's ID and name and record them in the import log.

In the onAfter script, the source and target data are both written to the import log. The script again iterates through the transformed values of the sn_etl_demo_worker table. Now that the data has been inserted into the target table, each record has a sys_id. The script records each worker's Sys ID, ID, and name in the import log.

Note:

• For creating these definitions and mapping using a guided or step-by-step experience, refer to Integration Hub - Import.
• If your target table or application is related to CMDB, use the IntegrationHub ETL guided experience.

Related tasks

• Define ETL entities
• Define ETL entity mappings
• Create ETL entity field definitions
• Define Robust Transform Engine operations