[SGC-AWS 2.0 will be available in stores on May 4, 2023]
Service Graph Connector for AWS Release 2.0 is packed with many new features listed below. This article describes the features in detail.
- Multi-Org (Multi-Instance) Support
- Single /Standalone Account Support
- Diagnostic Tool Update for Multi-Org and Standalone Account
- AWS Key Rotation
- Server Classification
- Lifecycle Stage and Lifecycle Stage Status
- Performance Improvements on Deleted Resource details
- Performance Improvements on Tagging API
AWS Setup instructions is now available as part of KB article (KB1220597).
1. Multi-Org (Multi-Instance) Support
SGC-AWS will now support multi-organization, referred to as "multi-instance" in SGC terms. This is similar to other SGC products. Before SGC-AWS 1.6.1, only Single-Org support was available, which required creating a ServiceNow user and setting up AWS Config and SSM Inventory in each AWS organization. To set up multi-instance, users will need to provide credentials and other AWS properties as part of the guided setup process.
- SGC-AWS environment properties are currently stored in the sys_properties table.
- The multi-instance upgrade will move these properties to the sn_aws_integ_sg_aws_application_properties table upon upgrading to version 2.0.
- A schedule and data source must be created for each organization or connection, similar to other SGC applications.
- Each organization-specific property can be configured through the "Configure AWS environment for the new Instance" option in guided setup.
| # | Property Section | Property Attributes | Description |
| 1 | Select the Credential Alias* | Connection Credentials (Access Key, Secret Key) | Need separate credential for each AWS Org. |
| 2 | Organization Details | Organisation Account* |
Organization Details describe the connection details. Organization Account ID. |
| 3 | Organisation Name* |
Organization Name if a free text for you to remember eg. Dev Org, QA Org, Prod Org.
Org Account ID & Name are used in the diagnostic tool to show the results. It is not used for any business logic. |
|
| 4 | Organisation Description | Its a free text to describe the AWS organization. | |
| 5 | AWS Regions | Select the AWS Regions you have created resources. If not selected, SGC-AWS will pull resource details from all the regions, which will impact performance. | |
| 6 | STS Assume Role* | STS Assume role created for ServiceNow user to assume in each account. Please enter only role name and not role ARN. | |
| 7 | Management Account ID |
If you have created ServiceNow user in a designated member account, enter Management Account ID. Leave blank if you have created ServiceNow user in Management Account. |
|
| 8 | AWS Config Aggregator Information | Config Aggregator Account ID | Account ID where the AWS Config Aggregator is created. |
| 9 | Config Aggregator Region | AWS Region where the AWS Config Aggregator is created. | |
| 10 | Config Aggregator Name | AWS Config Aggregator name where the AWS Config Aggregator is created. | |
| 11 | SSM S3 Bucket Details | S3 Account ID | Account ID where the S3 bucket is created. |
| 12 | S3 Region | AWS Region where the S3 bucket is created. | |
| 13 | S3 Bucket Name | Bucket name name where the S3 bucket is created. | |
| 14 | SSM Document Details | SSM Document Name for Linux. (Default Value - SG-AWS-RunShellScript) | SSM Document name for Linux Server packaged as part of the deployment scripts. If you have given a different document name, enter here. |
| 15 | SSM Document Name for Windows. (Default Value - SG-AWS-RunPowerShellScript) | SSM Document name for Windows Server packaged as part of the deployment scripts. If you have given a different document name, enter here. | |
| 16 | AWS Key Rotation Details | AWS Rotate Keys | Boolean flag (TRUE/FALSE) to rotate AWS Access Keys. By default it is set to FALSE. If set to TRUE, SG-AWS rotate keys with the duration period. |
| 17 | AWS Key Rotation Date | Date when the key is rotated lastly. Initially the date is empty. | |
| 18 | AWS Key Rotation Period (Default Value - 90 days) | Duration in days you want to rotate the keys. | |
| 19 | AWS Key Rotation Status | Status generated by SGC-AWS after each rotation. | |
| 20 | Standalone Account ID Setup | Standalone Account ID | If you are setting up the SGC-AWS on a single AWS account for evaluation or for security reasons, you need to give the account ID. |
* - Required Parameter.
Frequently Asked Questions
1. I have 3 AWS organizations, can I create one ServiceNow account and STS assume role in all the accounts in 3 organizations?
NO. It is best practice to create a ServiceNow user per organization and encapsulate its properties in one connection.
2. Our organization doesn’t support STS assume role between accounts, how can we configure SGC-AWS
For this use case, it is better to go with Single/Standalone setup.
2. Single / Standalone Account Support
Use Cases:
- Current SGC-AWS version supports AWS organizations with multiple member accounts and uses STS Assume role to access and retrieve CI data.
- Some clients have security restrictions that prevent STS Assume role to access other accounts.
- Clients want to evaluate SGC-AWS and replicate it to other AWS accounts.
- Solution consultants, who have only one account, were unable to showcase SGC-AWS to clients.
- The new release of SGC-AWS will support single accounts to address these issues.
Setup:
To set up a single account in SGC-AWS, you must provide the account ID in the "Configure AWS environment for the new Instance" option, located under the "Standalone Account ID Setup" section. This configuration will enable the application logic of SGC-AWS to support a single account.
- Provide the account ID in the "Configure AWS environment for the new Instance" option.
- Enter the account ID under the "Standalone Account ID Setup" section.
- This configuration enables the SGC-AWS application logic to support a single account.
Usecase Summary:
To create Cloud Org (cmdb_ci_cloud_org) and Cloud Service Account (cmdb_ci_cloud_service_account), the DescribeOrganization and ListAccount APIs are used in Single or multi-org scenarios. However, in a Standalone account scenario, these APIs are not invoked, and instead, the Standalone Account ID is populated in both the Cloud Org (cmdb_ci_cloud_org) and Cloud Service Account (cmdb_ci_cloud_service_account) tables.
- For creating Cloud Org and Cloud Service Account, DescribeOrganization and ListAccount APIs are used in Single or multi-org scenarios.
- In Standalone account scenario, the above-mentioned APIs are not invoked.
- Instead, Standalone Account ID is populated in both Cloud Org and Cloud Service Account tables.
3. Diagnostic Tool Update for Multi-Org and Standalone Account
The diagnostic tool for both Standalone Account and Multi-Org scenarios has been updated. The UI interface has been slightly modified to load the appropriate diagnostic tool context for each connection, which you need to select. By default, SSM and SSM Deep discovery tests will be executed, but if you haven't set them up, you can skip them by selecting the respective checkbox. All other functionality remains the same as in the previous release.
4. AWS Key Rotation
- A Key Rotation feature has been introduced in SGC-AWS with a default rotation period of 90 days.
- Customers can adjust the rotation period based on their enterprise policy.
- The Key Rotation feature is disabled by default and needs to be enabled to use it.
- A daily job executes this feature.
- If the rotation period has been reached and key rotation is enabled, the system will delete the existing AWS Access key and create a new one.
Here is the sequence diagram for the key rotation process:
To enable Key Rotation, it must be configured in the AWS environment settings. The configuration includes several properties that are stored in the Application Properties table (sn_aws_integ_sg_aws_application_properties):
- AWS Rotate Keys – Enabled / Disabled (default)
- AWS Key Rotation Date – Date on which key was rotation
- AWS Key Rotation Period - Default Value - 90 days
- AWS Key Rotation Status – Status of Key Rotation process
Once activated, the scheduled job selects the AWS credentials and replaces them with new ones. Email Notification Setup is also required to send emails, assuming that it has already been done by ServiceNow admins. The ServiceNow user must have AWS IAM iam:CreateAccessKey and iam:DeleteAccessKey permissions assigned. If the key rotation process is unsuccessful, the application will send a notification email.
5. Server Classification
- Before version 1.6.1, SGC-AWS populated server records in the parent cmdb_ci_server class.
- With this release, server records will be populated in either cmdb_ci_linux_server or cmdb_ci_win_server based on the server type obtained through SSM setup.
- If SSM setup is not enabled, the application will populate server records in the cmdb_ci_server class.
6. Life Cycle Stage and Life Cycle Stage Status
- SGC-AWS will now update the Lifecycle Stage and Status when the servers' status changes, with the introduction of the Lifecycle Stage and Status feature.
- The Lifecycle State and Lifecycle State Status will be updated accordingly when the operational status of virtual machines changes.
- When a virtual machine is turned off, the operational status changes to retired.
- The feature requires the installation and activation of the com.snc.cmdb.csdm.activation plugin.
- More information about the feature can be found in the Platform documentation provided link.
7.Performance Improvements on Deleted Resource details
SGC-AWS uses the AWS Config - Aggregator APIs to import CI data effectively into CMDB for newly created resources and updates. However, to obtain deleted information, we are forced to use the Account region level API - ListDiscoveredResources API. Since this API is only accessible at the account and region levels, SGC-AWS must loop through every account and region to collect this deleted data. The default retention duration for AWS Config is 7 years, and there is no date range available in the config:ListDiscoveredResources API to select a subset of data. The API will paginate and extract all the data from AWS Config. When resources are repeatedly built and deleted, AWS Config accumulates deleted data. Due to the inability to specify a date range, the config:ListDiscoveredResources API makes several thousand API calls to gather all the data, which has a significant impact on performance. We have submitted a feature request to the AWS Config team, asking them to include the information about removed resources in the aggregator API for AWS Config Central, where we can specify a date range and retrieve the specific subset of data we need.
As part of this release, we will collect the active resources in the CMDB and query the ListDiscoveredResources API to check if the resource is still active. We will then mark the resources as retired or absent. This will significantly reduce API calls and perform the job faster.
8.Performance Improvements on Tagging API
SGC-AWS relies heavily on AWS Config APIs to obtain AWS Tags for most resource types. However, for certain types such as S3, DynamoDB, ELB V1, and ELB V2, tag information is not available in the Config API. Therefore, we have been using the Tag API to obtain tag information. Unfortunately, when we call the Tag API with the resource type, it results in a fixed number of API calls, which can be problematic for customers with 1000+ accounts.
As part of this release, when a resource of type S3, DynamoDB, ELB V1, or ELB V2 is created or updated, its information will be stored in a temporary table called sg_aws_tags_request. We will then make the API call with the information from the sg_aws_tags_request table, thus reducing the number of limited API calls and completing the job faster.
Important note to customer who are upgrading from previous version of SGC-AWS to 2.0 version:
In the Data Source, the application updates the Last Run Datetime to execute business logic during the delta flow process. When upgrading from the previous version to SGC-AWS 2.0 version, these files will not be automatically updated due to the change in the file. Consequently, you may encounter the following error if the application is not upgraded correctly.
"Error executing script : TypeError: Cannot read property "connection_alias_id" from undefined (sys_script_include.d22e7bdbc0a8016500a18e024bfc9aa3.script; line 11)"
"Data was not loaded properly with data source: SG-AWS-Organization"
Solution:
To ensure a complete migration, it is necessary for you to accept the upgrade. By doing so, the connection alias ID will be set correctly, and the application should function normally.
