Amazon AWS Cloud Discovery Step by Step Configuration

[Narsing1]

This Article helps you to understand the concepts of Cloud Management , Discovery in detail and how it practically works.  You can use this as a base document for all your configurations with regards to Amazon AWS Cloud Discovery.

Similarly, the below configurations can also be useful for other Cloud applications such as Azure etc.,

 Note: The MID Server Installation is a bit different in Paris, but most of the remaining configurations are the same.   I will update this Article sometime later with the Paris MID configuration as well.

What you need:

• Amazon Account
• Servicenow Personal Instance

---------------------------------------------------------------------------------------------------------------------

Before you begin

• ●  Install all the required plugins for Discovery & Cloud Management

• ●  But I would recommend, you install all the ITOM plugins that includes all the below

  modules to have a complete knowledge on how these modules works.

  • ○  Discovery

  • ○  Service Mapping

  • ○  Orchestration

  • ○  Cloud Management

  • ○  Event Management

●  You can activate these plugins from developer.servicenow.com after you login with your credentials. [ i.e. Manage → Instance → Action → Activate Plugin ]

---------------------------------------------------------------------------------------------------------------------

Amazon EC2 Instance configuration

• ●  Create Amazon account and obtain the free subscription for 1 Year.

  ○ Precautions

  • You need to provide your credit/debit card in order to obtain this subscription, but

    it will not charge you anything for 1 year.

  • Make sure you are observing your instance Billing frequently

  • Make sure you are using free EC2 Instances only to avoid unexpected billing to

    your card

• ●  Go to Services -> EC2 - EC2 Dashboard

• ●  Click on “Running Instances”

• ●  Click on “Launch Instance” and check “Free tier only” option to show only free ec2

  instances list on the right panel

• ●  Choose “Microsoft Windows 2012 Server R2 Base” AMI and follow with the default

  options provided by Amazon

• ●  Create a Key pair and download the private key. Proceed further to Launch your

  Instance

• ●  Meanwhile, go to the Security groups and allow all traffic for inbound and outbound from

  anywhere. NOte: This is needed for all types of discovery from the MID Server and just

  for a demo purpose, you can enable all types of traffic.

• ●  Now, your instance is ready to connect. Click on Connect

• ●  Click on “Download Remote Desktop File”. Prior to that open your private key file which

  you downloaded previously and copy the complete text from that file.

• ●  Click on “GetPassword”, paste the text of your private key and click on “Decrypt

  Password”

• ●  Copy the “Password” text and proceed to click on the RDP file which you downloaded

  previously.

• ●  After you login to this machine, do the following.

• ●  Go to the command prompt, type “wf.msc” to open the windows firewall settings

• ●  Click on “Windows Firewall properties” and allow inbound connections for Domain, Private and Public Tabs. Apply all settings and close that window

• ●  Now, your instance is ready to install the MID Server. From this instance open your browser and access your Servicenow instance

---------------------------------------------------------------------------------------------------------------------

MID Server Configuration

• ●  Go to Mid Server → Downloads and click on 64-bit for Windows.

• ●  Create a folder on C:\ called “MID_SERVER”

• ●  Copy the extracted files to the above folder.

• ●  Open the Agent folder and click on “Installer.bat”

• ●  Provide your Servicenow instance url & Credentials. Click on Test connection and proceed next

• ●  Provide MID Server name and follow the next step.

• ●  Start the MID Server from here

• ●  You will be able to see an entry in your servicenow instance under MID Server → Server

• ●  Open that entry and click on “Validate” under Related Links

• ●  Once it got validated, Click on Upgrade MID to make sure you have the latest version

  installed on this Server.

• ●  You will be able to see a link called “Enable Credential less discovery”. Click on that as

  well to enable full capabilities for a discovery. This installs the nmap.

• ●  Make sure your mid server status is showing “Up” and Validate value to “Yes”.

---------------------------------------------------------------------------------------------------------------------

Another Amazon EC2 Instance Creation

• ●  Create another Amazon EC2 Instance in the same way how you created for the 1st instance.

• ●  Login to this instance and go to Server Manager [Start -> Server Manager]

• ●  Now, we will make this as a Web Server. Proceed to click on Add Roles and features

○ Note: You may also proceed without installing this if you are trying for only Horizontal

Discovery. But if you want to see how Service Mapping works, better make this Server to

provide at least one service. Here I am choosing a Web Server.

• ●  Check the appropriate feature and install it. Here you need to select IIS.

• ●  Go to the command prompt and type wf.msc. Allow all inbound traffic as you did for the

  previous instance.

○ Note: Since this is for demo purpose, you may allow everything. But in a typical enterprise environment, it shouldn’t be like this and you really don’t need to worry about these configurations.

● Now, you configured everything as far as this Demo is concerned

---------------------------------------------------------------------------------------------------------------------

From MID Server

• ●  Try to ping the 2nd instance that you created in the previous step from this Server.

●  You are all set if you are getting the reply from 2nd AMI

---------------------------------------------------------------------------------------------------------------------

From Servicenow Instance

• ●  Go to Discovery → Credentials and click on “New” → “Windows Credentials”

• ●  Provide the username and password of the 2nd instance here.

  • ○  The username would be “ADMIN\Administrator”

  • ○  Password is your 2nd instance password

• ●  PRovide a name to this. [ex: mywebserver_credentials]

• ●  Click on Test Credentials and make sure it succeeded.

  • ○  PRovide the ip address of 2nd instance i.e. web server

  • ○  Provide the mid server

  • ○  Port is 135.

---------------------------------------------------------------------------------------------------------------------

From AWS Console

• ●  Go to Services → IAM and click on “Users” on left panel

• ●  Add a new User and go to the “Security Credentials” Tab

• ●  Create Access Key and download the .csv file

• ●  Copy the 12 digit Account ID from “User ARN” value. This is your account id (ex:

  arn:aws:iam::​5XXXXXXXXXX4​:user/user1)

• ●  Go to the Dashboard and copy the IAM Users sign in link

●  Now, you are ready to create your service account in SNOW instance

---------------------------------------------------------------------------------------------------------------------

From Servicenow Instance

• ●  Go to the AWS Discovery → Credentials

• ●  Create a new Discovery credentials here ○ Name → Provide a name

  ○ AWS Account → Provide the Username from AWS account users list ○ Access ID → Provide this from the downloaded .csv file.
  ○ Secret Access Key → Provide this from the downloaded .csv file.
  ○

• ●  Go to Discovery Schedules and click on “Cloud Discovery”/ You can also access this from Cloud Management → Cloud Admin Portal → Manage → Discovery Schedules.

• ●  It opens up Discovery manager wizard in a separate Browser Tab/ on the right panel when you are using Admin Portal.

• ●  Follow the steps to configure your Service account here. You can obtain all the details from AWS account that you created in the previous section.

  • ○  Provider → AWS

  • ○  Schedule Name → Give some meaningful name for your discovery schedule

  • ○  Configure MID (this is optional). You may configure this later as well.

  • ○  Account ID → Provide the 12 digit account id

  • ○  Discovery Credentials → Select the Credential that you created.

  • ○  URL → Obtain this from AWS → IAM → Dashboard → IAM Users sign-in link

• ●  Click on Test Account and move to next step

• ●  Now, it discovers all the data centers available in this location

• ●  Click Next to find the VM’s. This is also optional and you can configure later

○ MID Server Selection Type → Choose the appropriate option

• ●  Provide the default schedule. You can change this later based on your requirement.

• ●  Click on Finish and Run. Now the discovery runs and show you the dashboard

• ●  Now, if you observe the Discovery Schedule, you will see 2 schedules like this

  • ○  One is the initial one with the name you have given while creating. This is basically for Data centers.

  • ○  Other one is “After Discovery” schedule and naming convention for this would be <name-VM schedule>. Open this schedule and add the ip ranges/quick ip range of your Amazon VM ip address for Web Server that you configured earlier.

  • ○  You may do a quick discovery from discovery schedule as well to check whether the VM discovery is coming / not

Note:  This Article helps you to understand how the AWS Cloud Discovery works in General .  You may need to customize based on your Organization Requirements.

[Vivek Verma]

Hi, Thank you for the article. 

Is the article works for Paris too? Because Mid Server Installation is not the same way, you describe in the article. 

Thanks, 

Vivek || Medium || LinkedIn

[Narsing1]

Hi,

Thank you for your feedback.  This was done on Orlando.  I will update the Article with Paris Release as well soon.  Remaining all the configurations would be the same in Paris as well, but I haven't tried in Paris release.

Thanks,

Narsing

[Runjay Patel]

Check out this video, it will clear all your doubts and help you to understand Cloud Discovery queries in details.

Link: https://www.youtube.com/watch?v=GWAvGbnCmlQ&ab_channel=ServiceNowHelpdesk

It help you to understand below points.

• Cloud Discovery
• Cloud Service Mapping vs Discovery
• Steps by step setup
• Practice with real world scenario
• Troubleshooting
• CI Class Tables and Attributes
• Amazon AWS Cloud Discovery Step by Step Configuration
• Azure Cloud Discovery Step by Step Configuration

Please mark reply as Helpful/Correct, if applicable. Thanks!!

[Long6]

why do we need this?

Open this schedule and add the ip ranges/quick ip range of your Amazon VM ip address for Web Server that you configured earlier.

I believe ServiceNow collects the IP Address automatically

[Narsing1]

Hi,

Not getting.  If your question relates to why we need all these steps,  then this Article is for the folks who are new to Cloud and Discovery and trying configure from the scratch.

I have mentioned this as a last point in the same article on how you can add the IP Ranges. 

Thanks,

Narsing

[Long6]

From SVNOW document, You do not need to configure IP ranges for the MID Servers used with Cloud Discovery. Discovery automatically selects the IP addresses of the virtual machines in the datacenters you selected in the wizard. 

Why do we need to do that step?

[Narsing1]

Correct me If I am wrong.  I think you are pointing this.

This schedule will be created automatically and if you want to limit your testing/for a quick testing to see how its working, then you may add, but yes you really don't need to add the ip ranges for AWS.  But the in initial Discovery schedule I didn't configure any IP Ranges.

Thanks,

Narsing

[Long6]

Yes, that's what I expected but for some reasons, it can't find any VMs in my AWS Resource; Discovery Schedule just throw an error saying that there is no IP Address associated with it.  Do you have any idea?

[Loehr25]

Thanks for the update and quick reply. I'll be sure to keep an eye on this thread.

omegle

[Greenholt]

After you configure the master account and supply the necessary credentials, you can test the connection to the account. If the test succeeds, Discovery returns a list of the member accounts in that master account. From this list, you can choose one or more sub-accounts to include in the Discovery of the master account. LiteBlue USPS

[Gentry62]

That's great. I was impressed by your writing. I am happy to see such a topic. Please come to my blog and read it.

DinarGuru

[Shiksha]

Thanks for the great post.

Bu when i tried implementing same for POC . I am getting an error that mid server is not able to communicate to other EC2 instance. Could you please suggest what could be issue.

[Narsing1]

Could you please post the error that you are getting?

Thanks,

Narsing

[Shiksha]

1)"Missing classification information" is discovery error i am facing when i am running AWS VM schedule.

2) For discovering second ec2 instance getting an error of ****** is not a reachable host (no response to target ports scanned by MID).

3)Have one query that what is difference between  horizontal  discovery working on subnets of  AWS cloud and cloud based aws discovery that AWS VM schedule that got created?

Kindly assist

[FernandoUrrutia]

My question is, what ports must open in aws for scan windows server with discovery... (is a range ports?).. for example.. 135 is the unique port? (updated on tokyo version)

[Shiksha]

No port required to be open As it is using cloud API to discover AWS cloud resources.

[Narsing1]

Hi @Shiksha 

Here you go.

• Have you tried to ping to the other EC2 Instance from the MID and if you are getting the Timeout error, then do this
  • Go to the Windows Firewall settings and create Inbound Rules and allow everything for testing purpose.
  • Go to the Security Groups in AWS Console and try to setup as per the article 
  • Are you able to access Internet from your MID? If not something to do with the Security Groups Inbound & Outbound Settings

• Have you moved from CAPI to Pattern based discovery? If not, please do that
• Do you have Discovery and Service Mapping patterns latest version
• Have you done the "Pattern Sync to MID"?
• Once you setup completely, On the Cloud Service Account - Are you able to discover the Data centers?

Thanks,

Narsing

[pratiksha5]

I have few questions. 

Do we need administrative access on aws side?

For vm discovery I am keep getting cancelled response due to IP range empty. 

The AWS amin says it is a master account however post running discovery servicenow is unchecking that option from service account

[Supriyarajput]

I need help with article on AWS discovery using iP address