AWS Service Graph Connector

Anne Marie Duff · ‎07-10-2024

How do I configure the AWS Service Graph Connector?

Version: 2.8

Sample Application Service to Monitor: MediaWiki

URL: http://172.31.43.163:5000/mediawiki

AWS Service Graph Connector Features

Enabling Deep Discovery on AWS EC2 Instances (Yes\No): Yes

Enabling Discovery on AWS EKS Clusters (Yes\No): No

AWS Environment Setup

Established Security Policy allows AWS IAM Users to be created in AWS Management Account: Yes

The ServiceNow IAM User will be created in the AWS Management Account. If the Established Security Policy does not allow AWS IAM Users to be created in the AWS Management Account then the ServiceNow IAM User is created in a Designated Member Account.

AWS Config Service already setup in AWS: No

AWS Systems Manager(SSM) already setup in AWS: No

AWS Systems Manager(SSM) Inventory already setup in AWS: No

AWS Systems Manager(SSM) Automations already setup for running across Multiple AWS Accounts from a

Central Account (Management Account\Delegated Administrator Account) : No

No of Member Accounts in AWS Organization: 1

No of Regions in Member Account: 1 -> us-east-1

ServiceNow Environment

Software Asset Management(SAM) Enabled on ServiceNow Instance (Yes\No): Yes

Application Environment

The MediaWiki Application is a 2-Tier PHP-based Web Application running on a 5 Linux EC2 Instance\VM Node configuration in our Crucible Lab AWS Environment which will be used to demonstrate the AWS Service Graph Connector.

A HAProxy Load Balancer (cruawslmwhap01) routes Web Requests to an Apache Web Server Cluster (cruawslmwapp01,cruawslmwapp02,cruawslmwapp03) that has the MediaWiki Application installed and running. The MediaWiki (PHP) Application in turn routes DB Requests to a MySQL Server DB (cruawslmwdb01) as depicted in the Top Down Discovery Service Map below.

The following topics are covered in this How do I configure the AWS Service Graph Connector? Article:

A. Installing AWS Service Graph Connector on your ServiceNow Instance

B. Configure AWS for monitoring your Application

C. Analyze your Application EC2 Instances in AWS

D. Configuring AWS Service Graph Connector on your ServiceNow Instance

E. Run AWS Service Graph Connector Scheduled Data Import Jobs on your ServiceNow Instance

F. Analyze the CMDB Records created\updated by the AWS Service Graph Connector for your Application in your ServiceNow Instance

G. When to use AWS Service Graph Connector vs Cloud Discovery

H. Large AWS Landscape Tuning and Troubleshooting - Large AWS Landscape Tuning

A. Installing AWS Service Graph Connector

(i) Login to your ServiceNow Instance

(ii) Install the following Application from the ServiceNow Store:

Service Graph Connector for AWS: sn_aws_integ

The following Applications are automatically installed\activated when you install this application

Discovery and Service Mapping Patterns: sn_itom_pattern
Integration Commons for CMDB: sn_cmdb_int_util
CMDB CI Class Model: sn_cmdb_ci_class

The following Plugins are automatically installed\activated when you install this application

Discovery Core: com.snc.discovery.core
Discovery - IP Based: com.snc.discovery.ip_based
ITOM Discovery License: com.snc.itom.discovery.license (Included with full Discovery Product)
ITOM Licensing: com.snc.itom.license
Pattern Designer (NG version): com.snc.ng.pattern.designer
ServiceNow IntegrationHub Action Template: com.glide.hub.action_type.datastream

(iii) Navigate to Setup under AWS in the Filter Menu and select it to bring you to Guided Setup

(iv) Go to the Configure the AWS environment - Download the scripts section

(v) Click Configure to Download a CloudFormationScripts-SG-AWS-v2.6 Zip file to your Desktop which contains the following AWS Cloud Formation Templates. The ones highlighted in Bold are going to be used in the next Configure AWS for monitoring your EC2 Instances Section:

EnableAWSConfig.yml - Enables AWS Config Service
AmazonSSMForInstancesRoleSetup.yml - Creates an AmazonSSMForInstancesProfile AWS Instance Profile
AWS-SystemsManager-AutomationAdministrationRole.yml - Creates AWS Role needed for running AWS System Manager Automations from a Central Account
AWS-SystemsManager-AutomationExecutionRole.yml - Creates AWS Role needed for running AWS System Manager Automation Documents
CreateServiceNowUser.yml - Creates AWS User needed for AWS Service Graph Connector Access
CreateSnowOrganizationAccountAccessRoleInMemberAccount.yml - Creates Role used for AssumeRole Access into Member Accounts
SG-AWS-RunKubeCtlEKSNamesShellScript.yml - Required when discovering AWS EKS Clusters
SG-AWS-RunKubeCtlShellScript.yml - Required when discovering AWS EKS Clusters
SG-AWS-RunPowerShellScript-Setup.yml - Deep Discovery on Windows EC2 Instances
SG-AWS-RunShellScript-Setup.yml - Deep Discovery on Linux EC2 Instances
SnowDesignatedAccountAccessRoleInManagementAccount.yml - Required when AWS ServiceNow User created in a Designated Member Account as oppose to the Management Account

Refer to the ServiceNow Executing scripts required for setting up AWS Documentation Page for more details.

B. Configure AWS for monitoring your Application

The steps below describe all the Setup that needs to be done in AWS before Configuring and Running the AWS Service Graph Connector.

1. AWS Config Service Setup

This step uses the EnableAWSConfig Cloud Formation Template listed in Section A. Installing AWS Service Graph Connector for setting up the AWS Config Recorder in our AWS Account. The AWS Config Recorder is a component of the AWS Config Service. The AWS Config Service keeps an Inventory of your AWS Resources. The AWS Config Recorder detects any changes in your AWS Resources in Near Real Time and updates the AWS Config Inventory with these changes. The AWS Service Graph Connector makes AWS Config API (config:ListDiscoveredResources) calls to the AWS Config Service to obtain Hardware information about your AWS Resources that is in turn used to populate Target Hardware CI's in the ServiceNow CMDB.

If the AWS Config Service along with the AWS Config Recorder is already setup in your AWS Account this step can be skipped.

(i) Log into your AWS Management Account

(ii) Create an EnableAWSConfig Cloud Formations Stack Set by uploading the EnableAWSConfig.yml file from your Desktop (Refer to the AWS Create a stack set Documentation Page for details on how to create an AWS Stack Set).

(iii) Specify the Region where your AWS Resources were created. The EC2 Instances associated with our MediaWiki Application were created in the us-east-1 Region so us-east-1 is specified in our case.

(iv) Click Next on all remaining screens where Default values for all fields on these screens are left as is.

(v) Click Submit after clicking on the Acknowledgement checkbox.

An EnableAWSConfig Stack will be created in every Account in your Organization for the Regions that you specify that in turn enables the AWS Config Service in those Accounts, Regions. For our example Use Case an EnableAWSConfig Stack will be created in our single Member Account for the us-east-1 Region with the AWS Config Service being enabled in the us-east-1 Region for this Member Account.

2. Create S3 bucket required for Deep Discovery - Deep Discovery Step

This step involves creating an S3 bucket needed for Deep Discovery. Enabling Deep Discovery (Please refer to the Deep discovery scripts section of the ServiceNow Executing scripts required for setting up AWS Documentation page for more information on Deep Discovery) on your AWS EC2 Instances means being able to obtain the Deep Discovery data listed below from your EC2 Instances:

Serial Number
Manufacturer
Model
Product Name
Process Information
TCP Connections
CPU Info

If you are not planning to enable AWS Deep Discovery for obtaining the above data from your EC2 Instances this step can be skipped.

(i) Log into the AWS Member Account that you have designated for storing temporary data associated with Deep Discovery on your EC2 Instances.

(ii) Switch to the Region that has the EC2 Instances that you want to do Deep Discovery on. e.g. us-east-1

(iii) Create a new S3 Bucket and make note of the Name you gave it. For example sgaws-bucket was used for our MediaWiki Application Use Case.

(iv) Leave the Default Field values as is on the Create Bucket Screen

(v) Click Submit to create the new S3 Bucket

An S3 Bucket, e.g. sgaws-bucket will be created in the Region you switched to, e.g. us-east-1. You will be providing this as the S3 Bucket Parameter in the Service Graph Connector for AWS Guided Setup.

3. Instance Profile Setup - Deep Discovery and SSM Manager Setup step

This step uses the AmazonSSMForInstancesRoleSetup Cloud Formation Template listed in section A. Installing AWS Service Graph Connector for creating the AmazonSSMForInstancesProfile AWS Instance Profile. This Instance Profile will contain the AWS Permissions needed for both SSM Systems Manager access and Deep Discovery. SSM Systems Manager Access is needed for collecting Software Inventory data from your EC2 Instances.

This step can be skipped for the following scenarios:

You are not planning to enable Deep Discovery for your EC2 Instances
You are not planning to collect Software Inventory data from your EC2 Instances
You are planning to collect Software Inventory data from your EC2 Instances but have already enabled SSM Manager Access for your EC2 Instances

(i) Log into your AWS Management Account

(ii) Create an AmazonSSMForInstancesRoleSetup Cloud Formations Stack Set by uploading the AmazonSSMForInstancesRoleSetup.yml file from your Desktop (Refer to the AWS Create a stack set Documentation Page for details on how to create an AWS Stack Set).

(iii) Specify the S3 Bucket Name associated with the S3 Bucket that you created in the previous Step, e.g. sgaws-bucket

(iv) Click Next until you get to the Set Deployment Options Screen where you specify Region.

- Specify a Region of your choosing.

(v) Click Next on all remaining screens where Default values for all fields on these screens are left as is.

(vi) Click Submit after clicking on the Acknowledgement checkbox.

An AmazonSSMForInstancesRoleSetup Stack will be created in every Account in your Organization that in turn creates a new AmazonSSMForInstancesRole Role in those Accounts. For our example Use Case an AmazonSSMForInstancesRoleSetup Stack will be created in our single Member Account with the AmazonSSMForInstancesRole Role created in that Member Account and an associated AmazonSSMForInstancesProfile being created for the Member Account.

The newly created AmazonSSMForInstancesRole Role will contain the below Policies:

AmazonSSMManagedInstanceCore - AWS Specific Policy that will allow your EC2 Instances to make calls to the Systems Manager (SSM) API.

Note: EC2 Instances make calls to the Systems Manager API via an SSM Agent that is installed on the Instances. In most cases SSM Agent is preinstalled on the Machine Image used to launch the Instance. If the Machine Image used to Launch the EC2 Instance did not contain SSM Agent then SSM Agent will need to be installed on your Instance. Please refer to this Amazon Amazon Machine Images (AMIs) with SSM Agent preinstalled Documentation Page for a list of Amazon Machine Images that contain SSM Agent.

SSMSendCommand - Policy that grants the below S3 Bucket permissions to the Role for allowing the Role access to the S3 Bucket that was created in the previous step.

- s3:PutObject
- s3:GetObject

- s3:PutObjectAcl

Note: The same permissions also need to be applied to the S3 Bucket itself to allow the Role access to the S3 Bucket. This is covered in the 5. Attach Bucket Policy to S3 Bucket Step below.

The newly created AmazonSSMForInstancesProfile Instance Profile will be attached to your EC2 Instances in the next step.

4. Attach Instance Profile to your EC2 Instances- Deep Discovery and SSM Manager Setup step

This step can be skipped for the following scenarios:

You are not planning to enable Deep Discovery for your EC2 Instances
You are not planning to collect Software Inventory data from your EC2 Instances
You are planning to collect Software Inventory data from your EC2 Instances but have already enabled SSM Manager Access for your EC2 Instances

(i) Log into your Member Account

(ii) Switch to the Region that has your EC2 Instances

(iii) Navigate to your EC2 Instances and for each Instance attach the AmazonSSMForInstancesProfile Instance Profile from the previous step to it. Please refer to the AWS Attach the Systems Manager instance profile to an instance (console) Documentation for details on how to do this.

(iv) Restart your Instances

Note: Restarting your EC2 Instances registers them with the AWS Systems Manager (SSM) Service.

5. Attach Bucket Policy to your S3 Bucket - Deep Discovery Step

This is one of the steps needed for enabling Deep Discovery. If you are not planning to enable AWS Deep Discovery for your EC2 Instances this step can be skipped.

A new Bucket Policy will be attached to your S3 Bucket that includes the AmazonSSMForInstancesRole Role from the previous Step 3. Instance Profile Setup.

(i) Login into your Member Account

(ii) Switch to the Region that you created your S3 Bucket in e.g. us-east-1.

(iii) Navigate to your S3 Bucket

(iv) Add a Bucket Policy like the one below to your S3 Bucket, substituting AccountNBR with your Member Account.

{
"Version": "2012-10-17",
"Id": "Policy123456789000",
"Statement": [
{
"Sid": "EC2S3Access",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountNBR:role/AmazonSSMForInstancesRole"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::sgaws-bucket/*"
}
]
}

(v) Save your changes

The AmazonSSMForInstancesRole Role, from the above 3. Instance Profile Setup step, now has permission to access your S3 Bucket.

6. SSM Send Command Document Setup - Deep Discovery Step

This step uses the SG-AWS-RunShellScript-Setup Cloud Formation Template listed in section A. Installing AWS Service Graph Connector and is one of the steps needed for enabling Deep Discovery. If you are not planning to enable AWS Deep Discovery for your EC2 Instances this step can be skipped.

For EC2 Instances in your AWS Account, Operating System Specific Commands can be executed on these Instances from AWS via the SSM Send Command API which takes an AWS SSM Document containing the Commands to run. The Output of these Commands are placed in the S3 Bucket specified in the Role that is executing the SSM Send Command Document. For the AWS Service Graph Connector the S3 Bucket is specified in the AmazonSSMForInstancesRole Role that was created in the above 3. Instance Profile Setup step. In our case it is the sgaws-bucket (that we created in the 2. Create S3 bucket required for Deep Discovery step)

Deep Discovery on your EC2 Instances\VMs is achieved by the AWS Service Graph Connector through the execution of AWS SSM Documents specific to the Operating System of your EC2 Instances. The Operating System specific Cloud Formation Scripts below are provided with the AWS Service Graph Connector (2 of the Cloud Formation Templates listed in section A. Installing AWS Service Graph Connector above).

SG-AWS-RunPowerShellScript-Setup.yml - Deep Discovery on Windows EC2 Instances
SG-AWS-RunShellScript-Setup.yml - Deep Discovery on Linux EC2 Instances

The AWS Service Graph Connector obtains the Command Output Deep Discovery data placed in the S3 Bucket and populates the respective Linux\Windows Server CI's in the ServiceNow CMDB with the Deep Discovery data listed below:

Serial Number
Manufacturer
Model
Product Name
Process Information
TCP Connections
CPU Info

As the Operating System associated with the EC2 Instances\VMs that our MediaWiki Application runs on is Linux, the SG-AWS-RunShellScript-Setup.yml Cloud Formation Script will be the Cloud Formation Script uploaded to AWS in the below steps.

(i) Login into your AWS Management Account

(ii) Create a SG-AWS-RunShellScript-Setup Cloud Formations Stack Set by uploading the SG-AWS-RunShellScript-Setup file from your Desktop.

(iii) Specify the Region where your AWS Resources were created. The EC2 Instances associated with our MediaWiki Application were created in the us-east-1 Region so us-east-1 is specified in our case.

(iv) Click Next on all remaining screens where Default values for all fields on these screens are left as is.

(v) Click Submit after clicking on the Acknowledgement checkbox.

An SG-AWS-RunShellScript-Setup Stack will be created in every Member Account in your Organization for the Regions that you specify that in turn creates an SG-AWS-RunShellScript SSM Document in those Accounts, Regions. For our MediaWiki Use Case an SG-AWS-RunShellScript-Setup Stack will be created in our single Member Account for the us-east-1 Region with a SG-AWS-RunShellScript SSM Document being created in the us-east-1 Region for this Member Account. This is the SSM Document that will be invoked by the AWS Service Graph Connector Send Command Scheduled Import Job listed in the E. Run AWS Service Graph Connector Scheduled Data Import Jobs on your ServiceNow Instance Section lower down.

7. ServiceNow User Setup

This step uses the CreateServiceNowUser Cloud Formation Template listed in section A. Installing AWS Service Graph Connector. It creates a ServiceNow User in either your Management Account or a Designated Member Account. For our MediaWiki Use Case we will be creating the ServiceNow User in the Management Account.

One of the Input Parameters that you will be providing to this CreateServiceNowUser Cloud Formation Template is Member Account Access Role Name, the Name of the Role that the newly created ServiceNow User will be assuming for accessing Member Accounts.

For our MediaWiki Use Case, a Custom SnowOrganizationAccountAccessRole Role will be created in a later 9. Member Account Access Role Setup Step that the ServiceNow IAM User will assume for accessing Member Accounts.

(i) Log into your AWS Management Account

(ii) Create a CreateServiceNowUser Cloud Formations Stack by uploading the CreateServiceNowUser file from your Desktop.

(iii) Stack Details Screen

User Name Field - Provide NOWSGCUser as input

Member Account Access Role Name field - Leave SnowOrganizationAccountAccessRole as the Default value

(iv) Click Next on all remaining screens where Default values for all fields on these screens are left as is.

(v) Click Submit after clicking on the Acknowledgement checkbox.

A CreateServiceNowUser Stack will be created in your AWS Management Account that in turn creates a NOWSGCUser IAM User in your AWS Management Account with the below Policies:

SnowAccountAccessPolicy - Provides NOWSGCUser User with Organization Permissions like e.g. List Organization Accounts (organizations:ListAccounts) that the User will use for obtaining the list of Member Accounts in the Organization etc.
SnowMemberAccountAccessPolicy - Provides NOWSGCUser User with permission to assume the SnowOrganizationAccountAccessRole Role in each Member Account (will be created in the 9. Member Account Access Role Setup step further down)

The NOWSGCUser User will be calling the AWS organizations:ListAccounts API in order to get the list of Member Accounts in the Organization. It will then be hopping into each of these Member Accounts using the SnowOrganizationAccountAccessRole Role to obtain information about the AWS Resources in these Accounts which will in turn be used to populate the corresponding Target CI's in the ServiceNow CMDB.

8. Create Access Key & Secret Access Key for ServiceNow User

(i) Navigate to the NOWSGCUser in the AWS IAM Users Module

(ii) Open the NOWSGCUser Record

(iii) Click on Create Access Key link on the NOWSGCUser Summary Screen that is displayed

(iv) Select the Third-party Service Radio button on the Screen that is displayed and click Next

(v) Click on the Create Access Key button on the Screen that is displayed

(vi) Save the Access Key and Secret Access Key that get generated. These will be provided as part of the AWS Service Graph Connector Guided Setup.

9. Member Account Access Role Setup

This step uses the CreateSnowOrganizationAccountAccessRoleInMemberAccount Cloud Formation Template listed in Section A. Installing AWS Service Graph Connector for creating the AWS Role that will be used for AssumeRole Access into the Member Accounts.

(i) Log into your AWS Management Account

(ii) Create a CreateSnowOrganizationAccountAccessRoleInMemberAccount Cloud Formations Stack Set by uploading the CreateSnowOrganizationAccountAccessRoleInMemberAccount file from your Desktop.

(iii) Stack Details Screen

Account Id Field - Provide your Management Account ID (We've created the ServiceNow User in the Management Account)

S3 Bucket Name Field - Provide the Name of the S3 Bucket you created in the 2. Create S3 bucket required for Deep Discovery step. In our MediaWiki Use Case it was sgaws-bucket.

ServiceNowUserName Field - Provide the Name of the ServiceNow User that was created in the above 7. ServiceNow User Setup step, NOWSGCUser.

(iv) Click Next until you get to the Set Deployment Options Screen where you specify Region

- Specify a Region of your choosing.

(v) Click Next on all remaining screens where Default values for all fields on these screens are left as is.

(vi) Click Submit after clicking on the Acknowledgement checkbox.

A CreateSnowOrganizationAccountAccessRoleInMemberAccount Stack will be created in every Member Account in your Organization that in turn creates a SnowOrganizationAccountAccessRole Role in those Accounts. For our MediaWiki Use Case a CreateSnowOrganizationAccountAccessRoleInMemberAccount Stack will be created in our single Member Account with a SnowOrganizationAccountAccessRole Role being created for this Member Account. It will have the following:

SnowOrganizationAccountAccessPolicy Policy- Provides Role with Resource Permissions like e.g. config:ListDiscoveredResources

Trust Relationship with the NOWSGCUser in the Management Account ID you specified in the Account Id Input Field

You will be providing this SnowOrganizationAccountAccessRole Role Name as the AssumeRole Parameter in Guided Setup. The NOWSGCUser will be assuming this SnowOrganizationAccountAccessRole Role which will in turn authorize it to be able to make the necessary AWS API Calls e.g. config:ListDiscoveredResources for obtaining AWS Resource data.

10. *AWS Systems Manager(SSM) Automations from Management Account Setup

AWS provides the ability to be able to run AWS System Manager Automations from a Central Account (Refer to the AWS Running automations in multiple AWS Regions and accounts Documentation Page for more details). We will be using the below Cloud Formation Templates, listed in Section A. Installing AWS Service Graph Connector, for creating the necessary AutomationAdministrationRole and AutomationExecutionRole Roles for enabling this.

AWS-SystemsManager-AutomationAdministrationRole
AWS-SystemsManager-AutomationExecutionRole

*This step can be skipped if you have no plans to run AWS System Manager Automations from a Central Account. If you instead plan to run AWS System Manager Automation Documents directly from individual Accounts you will only need to use the AWS-SystemsManager-AutomationExecutionRole Cloud Formation Template directly in these Accounts for creating the AutomationExecutionRole Role in these Accounts . The AutomationExecutionRole is needed for executing AWS System Manager Automation Documents.

(i) Log into your AWS Management Account

AWS-SystemsManager-AutomationAdministrationRole Stack

(ii) Create an AWS-SystemsManager-AutomationAdministrationRole Cloud Formations Stack by uploading the AWS-SystemsManager-AutomationAdministrationRole.yml file from your Desktop.

(iii) Click Next on all remaining screens where Default values for all fields on these screens are left as is.

(iv) Click Submit after clicking on the Acknowledgement checkbox.

A AWS-SystemsManager-AutomationAdministrationRole Stack will be created in your AWS Management Account that in turn creates a AWS-SystemsManager-AutomationAdministrationRole in your AWS Management Account. It will have the following:

AssumeRole-AWSSystemsManagerAutomationExecutionRole Policy - Allows the Role to Assume the AWS-SystemsManager-AutomationExecutionRole in the Member Accounts

AWS-SystemsManager-AutomationExecutionRole Stack Set

(v) Create an AWS-SystemsManager-AutomationExecutionRole Cloud Formations Stack Set by uploading the AWS-SystemsManager-AutomationExecutionRole.yml file from your Desktop.

(vi) Stack Details Screen

AdminAccountId Field - Provide your Management Account ID

(vii) Click Next until you get to the Set Deployment Options Screen where you specify Region

- Specify a Region of your choosing.

(viii) Click Next on all remaining screens where Default values for all fields on these screens are left as is.

(ix) Click Submit after clicking on the Acknowledgement checkbox.

An AWS-SystemsManager-AutomationExecutionRole Stack will be created in every Member Account in your Organization that in turn creates a AWS-SystemsManager-AutomationExecutionRole Role in those Accounts. For our MediaWiki Use Case a AWS-SystemsManager-AutomationExecutionRole Stack will be created in our single Member Account with an AWS-SystemsManager-AutomationExecutionRole Role being created for this Member Account. The AWS-SystemsManager-AutomationExecutionRole will have the following Policies:

AmazonSSMAutomationRole (Amazon Role) - Provides permissions for the SSM Automation service to execute activities defined within Automation documents
ExecutionPolicy - Grants the Role Permission to get EC2 Instance data

Note: You will need to attach the 2 Policies listed below, with associated Permissions, to this Role in order for Automation to work for the AWS-SetupInventory Automation Document referenced in the next step

IAM Policy

"iam:ListUserTags",
"iam:ListRoleTags",
"iam:TagUser",
"iam:TagRole",
"iam:UntagUser",
"iam:UntagRole",
"iam:CreateRole",
"iam:PutRolePolicy",
"iam:GetRole",
"iam:getRolePolicy",
"iam:DeleteRolePolicy",
"iam:DeleteRole"

Lambda Policy

"lambda:GetFunction",
"lambda:DeleteFunction",
"lambda:CreateFunction",
"lambda:InvokeFunction"

11. Run Inventory Automation - SSM Software Inventory Setup step

There is an AWS AWS-SetupInventory Automation Document that can be executed from your AWS Management Account for automatically creating an SSM Inventory in specified Regions in your Member Accounts. It triggers the execution of the same AWS-SetupInventory Automation Document in each Child Member Account. The AWS-SystemsManager-AutomationAdministrationRole in the AWS Management Account and AWS-SystemsManager-AutomationExecutionRole in each Member Account is required for doing this (created in the previous step).

This step can be skipped if you have already setup the SSM Systems Manager Software Inventory in your AWS Account.

(i) Log into your AWS Management Account

(ii) Navigate to the AWS Automation Module and execute the AWS-SetupInventory Automation Document

(iii) Select the Multi-Account and Region Radio Button

(iv) Specify the Member Account Numbers and Regions that you want to run the automation in

- In our MediaWiki Use Case it will be our Single Member Account with the us-east-1 Region

(v) For the AutomationAssumeRole Input Parameter select the AWS-SystemsManager-AutomationAdministrationRole Role from the Pulldown Menu

(vi) Click on Execute to kick off the Automation

(vii) At the end of the Execution log into your Member Accounts to verify that SSM Inventory has been successfully setup for the specified Regions.

You should see an SSM Inventory Dashboard like the one below for the Region that you specified if the AWS-SetupInventory Automation Execution was successful.

Note: If you do not want to execute the AWS-SetupInventory Automation Document from a Central Account that triggers the execution of the same AWS-SetupInventory Automation Document in each Member Account you can execute the AWS-SetupInventory Automation Document from each Member Account itself where you select the Simple Execution Radio button instead of the Multi-Account and Region Radio Button and click on the Execute Button. You will need to have the AutomationExecutionRole Role created in these Member Accounts (as mentioned in the previous step). The SSM Inventory will be created for your Member Account in the Region that you executed the Automation Document from.

C. Analyze your Application EC2 Instances in AWS

The 2 AWS Modules below are used to Manage EC2 Instances in AWS:

EC2 - Start/Stop Instances, EC2 Instance Properties
Systems Manager Inventory - Metadata associated with Resources like e.g. Applications, Network Adatpers installed on the EC2 Instances

EC2

The AWS EC2 Module manages the EC2 Instances associated with an AWS Account.

(i) Log into your AWS Member Account

(ii) Switch to the Region that has your EC2 Instances

(iii) Navigate to EC2, Instances to see the list of EC2 Instances associated with the Region you switched to in your Member Account.

Below is an example of this for the 5 EC2 Instances\VM's associated with the MediaWiki Application in our Crucible Lab Environment:

(iii) Click on any of the EC2 Instances to bring up the Details associated with the EC2 Instance. You will see

a Details Screen for the Instance with the following Additional Tabs:

Networking
Storage
Tags

Below is an example of this for our cruawslmwapp01 EC2 Instance where the details associated with our cruawslmwapp01 EC2 Instance\VM is displayed.

Network Interfaces

(iv) Click on the Networking Tab to bring up the list of Network Interfaces on your EC2 Instance\VM. Below is an example of this for our cruawslmwapp01 EC2 Instance\VM where the list of Network Interfaces on the cruawslmwapp01 EC2 Instance\VM is displayed.

Storage Volumes

(v) Click on the Storage Tab to bring up the list of Storage Volumes on your EC2 Instance\VM. Below is an example of this for our cruawslmwapp01 EC2 Instance\VM where the list of Storage Volumes on the cruawslmwapp01 EC2 Instance\VM is displayed.

Add Multi Instances

There is an Add Multi Instances section in Guided Setup that is not Mandatory but is recommended even if you are only using One AWS Organization. It allows you to create a set of Data Sources and Scheduled Imports that are specific to your Customer Specific AWS Organization. This is recommended for the following reasons:

It is good futureproofing for cases where you may need to connect to a 2nd AWS Organization sometime in the future. For example an AWS Organization in a different company that is acquired through corporate M&A activity or for a case when you want to connect to AWS Gov Cloud vs AWS Commercial.

It prepares you for future upgrades, where the Customer specific Data Source and Scheduled Data Import Records in the sys_data_source Table will not be marked as Skipped Records for Review by the Upgrade. It will allow you to focus on Skipped Records due to intentional Customization as oppose to Execution of the Out of the Box Scheduled Imports.

Note: It is also recommended as a Best Practice for handling Large AWS environments that have a Large number of Member Accounts with associated Regions where the Performance of some of the AWS Scheduled Import Jobs is impacted, the Performance associated with the Load Times of these jobs in particular. (This is explained in more detail in the Large AWS Landscape Tuning and Troubleshooting - Large AWS Landscape Tuning Article).

Go through all the steps in this Add Multi Instances section to specify Customer specific Data Sources and Scheduled Imports. Pay particular attention to the below steps in this section:

Create new Connection & Credentials Alias Record Step

(i) Click on the Configure button to the right of Create a new Connection & Credentials Alias Record to bring up the Workflow Studio.

(ii) Click on the Integrations Tab and click Add Connection on the SG-AWS-CredentialAlias-Org (Parent Connection & Credential Alias) Connection Tile to bring up the below Create Connection Dialog Box:

Connection name: Enter a Name that will allow you to easily identity the AWS Data Source that you are connecting to, e.g. USA. This Name will be used as part of the naming convention for the newly created Customer Specific Data Sources & Import Sets as per below.

Customer Specific Data Sources	Connection Name - Data Source Name
Customer Specific Scheduled Import Jobs	Connection Name - Import Job Name

Access Key ID, Secret Access Key: Specify the Access Key ID & Secret Access Key associated with the Customer specific AWS Data Source that you are connecting to.

(iii) Click on Create Connection

A new Child Connection & Credential Alias Record is created with the Access Key ID & Secret Access Key values specified. This Child Connection & Credential Alias Record is associated with the Parent "SG-AWS-CredentialAlias-Org" Connection & Credential Alias Record as shown in the below screen shot. We specified USA for our example so USA is shown as the Child Connection & Credential Alias below.

A new set of Customer specific Data Sources and Scheduled Imports are created that contain the Connection Name specified in the Create Connection Dialog Box. An example of the Customer specific Data Sources and Scheduled Imports that get created is shown below, where USA was used to identify our Customer specific Schedule Imports and Data Sources:

Configure AWS environment for the new Instance Step

(i) Click on Configure to the right of Configure AWS environment for the new Instance to bring up the SG-AWS Configuration Properties Screen.

(ii) Populate the Connection Alias Field with the Child Connection & Credential Alias Record that was created in the previous step, e.g. sn_aws_integ.USA

(iii) For the remaining Fields on this screen, specify the properties associated with the new Customer Specific AWS Instance similarly to how you did with in the above 2. Update configuration properties for instance step for the Default AWS Instance

Configure the Scheduled Imports Step

(i) Open the Customer Specific Organization Scheduled Import Record, e.g. USA-SG-AWS-Organization shown in the above screen shot.

(ii) Select Active to activate the scheduled job.

(iii) Select Periodically for how often the job will run.

E. Run AWS Service Graph Connector Scheduled Data Import Jobs on your ServiceNow Instance

Before running these Scheduled Data Imports, I would recommend enabling CMDB 360 by setting the glide.identification_engine.multisource_enabled system property to True in System Properties.

Doing this allows the following for CI's that are Created\Updated by the Scheduled Data Import Jobs:

1. For CI's that have Reconciliation Rules, see Proposed Values for Lower Priority Discovery Sources that were Rejected
2. For CI's that allow more than 1 Discovery Source to update them (i.e. No Reconciliation Rules or Reconciliation Rules with same Priority), Identify the Source of an Attribute and see the Proposed Values for that Attribute from the other Discovery Sources.

Refer to the ServiceNow CMDB 360/Multisource CMDB Documentation page for more details.

(i) Navigate to Scheduled Data Imports under AWS in the Filter Menu. 31 Scheduled Data Imports should be listed, with all of them being marked Active as shown below. The Order Column shows the Order that the Import Jobs will run in.

Note: It is recommended that you create a Customer Specific Version of these Scheduled Data Imports as discussed in the Add Multi Instances Step of the above D. Configuring AWS Service Graph Connector on your ServiceNow Instance section.

(ii) For the Optional Scheduled Import Jobs (listed in the Scheduled Import Table in the Configure Scheduled Import Jobs step of the above D. Configuring AWS Service Graph Connector on your ServiceNow Instance section) that you have decided not to run, mark them as Active=false.

Open your SG-AWS-Organization Parent Scheduled Import job record and click on the Execute button

(iii) Navigate to Concurrent Import Sets in the Filter Menu.

- Wait for your Active Scheduled Data Import jobs to finish.

F. Analyze the CMDB Records created\updated by the AWS Service Graph Connector for your Application in your ServiceNow Instance

There are 5 types of Records created by the AWS Service Graph Connector in the CMDB:

CMDB CI[cmdb_ci] Records
Software Installation[cmdb_sam_sw_install] Records - If Software Asset Management(SAM) enabled
Software Instance[cmdb_software_instance] + Software Package[cmdb_ci_spkg] Records - If Software Asset Management(SAM) not enabled
Running Process[cmdb_running_process] Records
TCP Connection[cmdb_tcp] Records
Key Value[cmdb_key_value] Records

CMDB CI Records

(i) Navigate to cmdb_ci.list in the Filter Menu

(ii) Group by Discovery Source

(iii) Navigate to the SG-AWS Discovery Source and double click on its Discovery source:SG-AWS(n) link where n represents the Number of CMDB records(entities) Created\Updated by the SG-AWS Service Graph Connector.

(iv) Group By Class

A List of CMDB CI Records Created\Updated by the SG-AWS Service Graph Connector will be displayed grouped by Class. The screen shot below shows some of the Class Records (not all fit in 1 screen) displayed in this Class List for the data that was ingested by the SG-AWS Service Graph Connector for our AWS Instance that includes CI's associated with our MediaWiki Application:

Note: For ServiceNow Instances that do not have Software Asset Management(SAM) enabled, you would see an extra Software Class listed for representing all the Software Package[cmdb_ci_spkg] Records that would have been populated by the SG-AWS Software Inventory Scheduled Data Import Job (as referenced in the Scheduled Data Import Job Table from above D. Configuring AWS Service Graph Connector on your ServiceNow Instance section).

Notice 3 Apache Web Server Records to represent the 3 cruawslmwapp01, cruawslmwapp02, cruawslmwapp03 Apache Web Servers in our MediaWiki Application Topology
Notice 1 MySQL Instance Record to represent our cruawslmwdb01 MySQL DB Server in our MediaWiki Application Topology
Notice 1 HAProxy Load Balancer Record to represent the cruawslmwhap01 HA Proxy Load Balancer in our MediaWiki Application Topology

The 5 Linux EC2 Instances associated with our MediaWiki Application are listed as Linux Server CI's as shown in the below screenshot:

The Network Adapter's, IP Addresses, Storage Volumes and Network Interface Cards associated with each of these Linux Servers are listed as Network Adapter, IP Address, Storage Volume and Cloud Mgmt Network Interface CI Classes respectively as shown in the below screen shot:

These were populated in turn from the corresponding Network Adapter, IP Address, Storage Volume and Network Interface Card Entities in AWS. The Network Adapters, IP Addresses, Storage Volume and Network Interface Cards associated with the cruawslmwapp01 EC2 Instance in AWS are shown in the above C. Analyze your Application EC2 Instances in AWS Section.

cruawslmwapp01 Linux Server

The screen shot below shows all the Linux Server Summary fields that were populated by the connector for the cruawslmwapp01 Linux Server CI created by the AWS Service Graph Connector. The Serial Number, Model ID, Manufacturer, RAM and CPU fields that were populated by the Deep Discovery Feature are circled.

Related Tabs

The screen shot below shows the Network Adapter(2) and CI IPs(2) Tabs that were populated with the Network Adapters and CI IP Records associated with the cruawslmwapp01 Linux Server CI respectively. For example the 2 Network Adapter Mac Addresses shown can be seen for this cruawslmwapp01 EC2 Instance under Network Adapters in the C. Analyze your Application EC2 Instances in AWS Section.

Note: The 120 Processes in the Running Process(120) Tab and the 6 TCP Connections in the TCP Connections(6) Tab were populated by Deep Discovery with the 482 Software Installed Records being populated by the Software Inventory Import Job.