Data mapping for Service Graph Connector for AWS
Data from the AWS data sources is mapped and transformed into the ServiceNow CMDB Configuration Item (CI) class definitions using the Robust Transform Engine (RTE). Data is inserted into the ServiceNow® CMDB using the Identification and Reconciliation Engine (IRE).
Data mapping for AWS
Data from the AWS data sources is mapped and transformed into the ServiceNow CMDB Configuration Item (CI) class definitions using the Robust Transform Engine (RTE). Data is inserted into the ServiceNow CMDB using the Identification and Reconciliation Engine (IRE).
| Order | Name (data source or import schedule) |
Staging table | CMDB CI classes | Import schedule requirement type |
Import schedule dependencies |
|---|---|---|---|---|---|
| 1 | SG-AWS-Organization | SG-AWS-Organization [sn_aws_integ_sg_aws_organization] |
Required |
None |
|
| 2 | SG-AWS-Org-Units | SG-AWS-Org-Units [sn_aws_integ_sg_aws_org_units] |
Optional |
SG-AWS-Organization |
|
| 3 | SG-AWS-Service-Account | SG-AWS-Service-Account [sn_aws_integ_sg_aws_service_account] |
Required |
SG-AWS-Organization |
|
| 4 | SG-AWS-Service-Account-Tags | SG-AWS-Service-Account-Tags [sn_aws_integ_sg_aws_service_account_tags] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account |
|
| 5 | SG-AWS-Org-Unit-Accounts | SG-AWS-Org-Unit-Accounts [sn_aws_integ_sg_aws_org_unit_accounts] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account |
|
| 6 | SG-AWS-Datacenters | SG-AWS-Datacenters [sn_aws_integ_sg_aws_datacenters] |
Required |
SG-AWS-Organization SG-AWS-Service-Account |
|
| 7 | SG-AWS-VPC | SG-AWS-VPC [sn_aws_integ_sg_aws_vpc] |
SG-AWS Extension Attributes |
Required |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
| 8 | SG-AWS-Subnets | SG-AWS-Subnets [sn_aws_integ_sg_aws_subnets] |
SG-AWS Extension Attributes |
Required |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC |
| 9 | SG-AWS-Network-Interface | SG-AWS-Network-Interface [sn_aws_integ_sg_aws_network_interface] |
SG-AWS Extension Attributes |
Required for a virtual machine (VM) instance |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC |
| 10 | SG-AWS-Security-Group | SG-AWS-Security-Group [sn_aws_integ_sg_aws_security_group] |
SG-AWS Extension Attributes |
Required for a VM instance |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC |
| 11 | SG-AWS-Storage-Volume | SG-AWS-Storage-Volume [sn_aws_integ_sg_aws_storage_volume] | Storage Volume SG-AWS Extension Attributes |
Required for a VM instance |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC |
| 12 | SG-AWS-Image-Private | SG-AWS-Image [sn_aws_integ_sg_aws_image] |
Required for a VM instance |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC |
|
| 13 | SG-AWS-Image-Id | SG-AWS-Image-Id [sn_aws_integ_sg_aws_image_id] |
Required for a VM instance |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC |
|
| 14 | SG-AWS-Hardware-Type | SG-AWS-Hardware-Type [sn_aws_integ_sg_aws_hardware_type] |
Required for a VM instance |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC |
|
| 15 | SG-AWS-EC2 | SG-AWS-EC2 [sn_aws_integ_sg_aws_ec2] | The following CIs are populated when populating the Virtual Machine Instance CI: |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Hardware-Type SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id |
| 16 | SG-AWS-ELB-V1 | SG-AWS-ELB-V1 [sn_aws_integ_sg_aws_elb_v1] |
SG-AWS Extension Attributes |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
| 17 | SG-AWS-ELB-V2 | SG-AWS-ELB-V2 [sn_aws_integ_sg_aws_elb_v2] |
SG-AWS Extension Attributes |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
| 18 | SG-AWS-RDS | SG-AWS-RDS [sn_aws_integ_sg_aws_rds] |
SG-AWS Extension Attributes |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
| 19 | SG-AWS-API-Gateway | SG-AWS-API-Gateway [sn_aws_integ_sg_aws_api_gateway] |
Cloud Gateway [cmdb_ci_cloud_gateway] SG-AWS Extension Attributes |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
| 20 | SG-AWS-Lambda | SG-AWS-Lambda [sn_aws_integ_sg_aws_lambda] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
|
| 21 | SG-AWS-S3 | SG-AWS-S3 [sn_aws_integ_sg_aws_s3] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
|
| 22 | SG-AWS-DynamoDb | SG-AWS-DynamoDb [sn_aws_integ_sg_aws_dynamodb] |
SG-AWS Extension Attributes |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
| 23 | SG-AWS-Software-Inventory | SG-AWS-Software-Inventory [sn_aws_integ_sg_aws_software_inventory] SG-AWS-Software-Staging [sn_aws_integ_sg_aws_temp_software_staging] |
When the Software Asset Management (SAM) application isn't installed: When the SAM application is installed: |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Hardware-Type SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 |
| 24 | SG-AWS-Software-Remove | SG-AWS-Software-Remove [sn_aws_integ_sg_aws_software_remove] |
None |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Hardware-Type SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 SG-AWS-Software-Inventory |
| 25 | SG-AWS-SSM-SendCommand | SG-AWS-SSM-SendCommand [sn_aws_integ_sg_aws_ssm_sendcommand] |
Running Process [cmdb_running_process] TCP Connections [cmdb_tcp] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 |
| 26 | SG-AWS-Tags | SG-AWS-Tags [sn_aws_integ_sg_aws_tags] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-ELB-V1 SG-AWS-ELB-V2 SG-AWS-DynamoDb SG-AWS-Lambda |
|
| 27 | SG-AWS-VM-Hw-Consolidation | SG-AWS-VM-Hw-Consolidation [sn_aws_integ_sg_aws_vm_hw_consolidation] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 SG-AWS-Hardware-Type |
|
| 28 | SG-AWS-EKS-Cluster | SG-AWS-EKS-Cluster [sn_aws_integ_sg_aws_eks_cluster] |
SG-AWS Extension Attributes |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 SG-AWS-Hardware-Type SG-AWS-VM-Hw-Consolidation |
| 29 | SG-AWS-EKS-Cluster-2 | SG-AWS-EKS-Cluster-2 [sn_aws_integ_sg_aws_eks_cluster_2] |
SG-AWS Extension Attributes |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 SG-AWS-Hardware-Type SG-AWS-VM-Hw-Consolidation SG-AWS-EKS-Cluster |
| 30 | SG-AWS-EKS-FULL | SG-AWS-EKS-FULL [sn_aws_integ_sg_aws_eks_full] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 SG-AWS-Hardware-Type SG-AWS-VM-Hw-Consolidation SG-AWS-EKS-Cluster SG-AWS-EKS-Cluster-2 |
|
| 31 | SG-AWS-Generic-Resources | SG-AWS-Generic-Resources [sn_aws_integ_sg_aws_generic_resources] |
SG-AWS Extension Attributes |
Optional |
SG-AWS-Organization |
| 32 | SG-AWS-Redshift-Cluster | SG-AWS-Redshift-Cluster [sn_aws_integ_sg_aws_redshift_cluster] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
|
| 33 | SG-AWS-Get-Inventory | SG-AWS-Get-Inventory [sn_aws_integ_sg_aws_get_inventory] |
Required |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters |
|
| 34 | SG-AWS-GenericTags | SG-AWS-GenericTags [sn_aws_integ_sg_aws_generictags] | Optional |
SG-AWS-Organization SG-AWS-Generic-Resources |
|
| 35 | SG-AWS-SendCommand | SG-AWS-SendCommand [sn_aws_integ_sg_aws_ssm_sendcommand] | None 주: The SG-AWS-SendCommand data source doesn't have target CMDB CI classes. This data source populates the data into the sn_aws_integ_sg_aws_ssm_sendcommand staging table, but the import records aren't
transformed, and the import sets remain in pending state. |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 |
| 36 | SG-AWS-SSM-GetS3Object | SG-AWS-SSM-GetS3Object [sn_aws_integ_sg_aws_ssm_gets3object] |
Running Process [cmdb_running_process] TCP Connections [cmdb_tcp] |
Optional |
SG-AWS-Organization SG-AWS-Service-Account SG-AWS-Datacenters SG-AWS-VPC SG-AWS-Subnets SG-AWS-Network-Interface SG-AWS-Security-Group SG-AWS-Storage-Volume SG-AWS-Image-Private SG-AWS-Image-Id SG-AWS-EC2 SG-AWS-SendCommand |
For more information on where data is saved when pulling data from AWS, see CMDB classes targeted in Service Graph Connector for AWS and Supported AWS resource types.
You can use the IntegrationHub ETL app to view the data maps. See IntegrationHub ETL for more information.
For more information about how CI information is pulled from AWS, see the Service Graph Connector for AWS - Functional Spec and CI article on the ServiceNow Community site.
- The SG-AWS-Generic-Resources data source imports data for generic resources that aren't tracked by other data sources. The connector uses the Service Graph Resource Inclusion Whitelist
[sn_cmdb_int_util_service_graph_resource_inclusion_whitelist] table to differentiate between generic and other supported resource types.
The connector first populates all supported resources in the Service Graph Resource Inclusion Whitelist [sn_cmdb_int_util_service_graph_resource_inclusion_whitelist] table. These resources, categorized under their respective supported resource types, have specific data sources designated for ingestion within the connector. When the SG-AWS-Generic-Resources data source is executed and retrieves unsupported resource types, they are added to the Service Graph Resource Inclusion Whitelist [sn_cmdb_int_util_service_graph_resource_inclusion_whitelist] table and categorized as generic.
- In Service Graph Connector for AWS version 2.13.0 and later, parallel loading is enabled for SG-AWS-Generic-Resources data source if the AWS Config aggregator is used for discovery. Partitions are created for the SG-AWS-Generic-Resources data source based on the Resource Types in the Service Graph Resource Inclusion Whitelist
[sn_cmdb_int_util_service_graph_resource_inclusion_whitelist] table. Additionally, delete jobs aren't created for a full data load.
If the AWS Config aggregator isn't used for discovery, API calls are made to multiple accounts and the Partition info field is populated with Account and Region details.
- To import global generic resources such as IAM user and IAM group, specify a standard AWS region that has Config enabled with includeGlobalResourceTypes
set to true by updating the value of the sn_aws_integ.global_generic_resource_region system property for the SG-AWS-Generic-Resources data source.
If an aggregator is configured, and the sn_aws_integ.global_generic_resource_region value is not specified, the aggregator region is assigned as the value of this system property for importing global generic resources.
- If the AWS Systems Manager (SSM) service isn't enabled, the connector populates the server records in the Server [cmdb_ci_server] class. If the AWS SSM service is enabled, then based on the platform type obtained through the SSM service, the server records are populated in either the Linux Server [cmdb_ci_linux_server] class or the Windows Server [cmdb_ci_win_server] class. The Server [cmdb_ci_server] class is the parent class of the Linux Server [cmdb_ci_linux_server] and the Windows Server [cmdb_ci_win_server] classes.
- All labels associated with an AWS resource are added to the Key Value [cmdb_key_value] table.주:You can use the CMDB Data Manager to delete tag data from retired CIs in the Key Value [cmdb_key_value] table based on conditions like retention time and discovery source. A scheduled job runs the policy, which can be configured to execute during off-peak hours.
- The basic information about an AWS resource is stored in the SG-AWS Extension Attributes [sn_aws_extension_attributes] table.
- In Service Graph Connector for AWS version 2.10.0 and later, the SG-AWS-Get-Inventory data source runs before the SG-AWS-EC2 data source and creates a Server [cmdb_ci_server] CI with the host name mapped to the Name attribute, instead of being mapped to the VM name.
- The SG-AWS-GenericTags data source imports tag data only for generic resources that have an ARN key. You can use the SG-AWS Extension Attributes [sn_aws_extension_attributes] table to verify which generic resources have an ARN key.
- The AWS configuration data for each connection is stored in the SG AWS Application Properties [sn_aws_integ_sg_aws_application_properties] table.
- When you run the diagnostic test, the data is loaded in the following tables:
- SG AWS Diagnostic Details [sn_aws_integ_sg_aws_diagnostic_details]
- SG-AWS Diagnostic Summary [sn_aws_integ_sg_aws_diagnostic_summary]
- SG AWS Diagnostic Summary Notes [sn_aws_integ_sg_aws_diagnostic_summary_notes]
- In Service Graph Connector for AWS version 2.13.0 and later, the SG-AWS-Image-Id data source doesn't access cross-account records during lookup.