Service Graph Connector for Azure - Overview

[vaibhavbhatnaga]

Overview

SG-Azure is part of ServiceNow developed Service Graph Connectors. The connector is built to simplify the onboarding setup and ease the integration with Azure with the minimum invasive principal. This article describes the concepts and setup of the Service Graph Connector for Azure. 

Objectives of SG-Azure –

• Easy to setup
• Should be able to complement the ServiceNow Azure cloud discovery
• Pulls the software details from Azure VMs
• Mid server isn’t required to setup

Azure components and API used

Change Tracking and Inventory

Change Tracking Inventory feature tracks changes in virtual machines hosted in Azure, on-premises, and other cloud environments to help you pinpoint operational and environmental issues with software managed by the Distribution Package Manager

Change Tracking and Inventory natively tracks:

• Software changes
• Windows services
• Linux daemons

** Change Tracking and Inventory Overview

Enabling all features included in Change Tracking and Inventory might cause additional charges. Before proceeding, review Automation Pricing and Azure Monitor Pricing

Azure Graph API

Importing Hardware Cis from Azure

Azure Log Analytics

Log Analytics is a tool in the Azure portal to edit and run log queries from data collected by Azure Monitor logs and interactively analyze their results. We query Azure log analytics workspace to get software information.

Setup on Azure

**Icon source https://azure.microsoft.com/en-in/

Step 1: Create a service principal (i.e. the authentication mechanism)

 Steps to register an app: 

• Sign into the Azure portal.
• Search for and select Azure Active Directory.
• Under Manage, select App registrations > New registration.
• Enter a display Name for your application. 
• Specify who can use the application.
• Select Register to complete the initial app registration.

• Click on the application and select ‘New client secret’  
• Record the secret's value for use in your client application code. This secret value is never displayed again after you leave this page.
• Select API permissions.
• Assign ‘User.Read’ permission in Microsoft Graph API with type ‘Delegated’.
• Assign ‘Data.Read’ permission in Log Analytics API with type ‘Delegated’.
• Select Subscription and select IAM
• Add the application you have created. 

Step 2: Create a Log Analytics Workspace

Created workspace for each region under each subscription and noted down the workspace id. Workspace id is required on guided setup for “Create connection for the software import”

Create a Workspace step and note down the workspace id.

Step 3: Create Automation Account

Steps for creating automation account for each region under each subscription

Step 4: Enable Change Tracking and Inventory from an Automation account

Step to enable change tracking          

Authentication Flow

1. The client application authenticates to the Azure AD token issuance endpoint and requests an access token.
2. The Azure AD token issuance endpoint issues the access token.
3. The access token is used to authenticate the secured resource.
4. Data from the secured resource is returned to the client application.

 Main flow

CI Classes

CI	Table	Azure Terminology
Cloud Service Account	cmdb_ci_cloud_service_account	Subscriptions
Logical Datacenter	cmdb_ci_logical_datacenter	Locations
Availability Zone	cmdb_ci_availability_zone	Availability Sets
Resource Group	cmdb_ci_resource_group	Resource Groups
Cloud Network	cmdb_ci_network	Virtual Networks
Cloud Subnet	cmdb_ci_cloud_subnet	Virtual Networks
Storage Volume	cmdb_ci_storage_volume	Disks
Compute Security Groups	cmdb_ci_compute_security_group	Network Security Groups
Servers	cmdb_ci_server	Virtual Machines
Virtual Server	cmdb_ci_vm_instance	Virtual Machines
Hardware Type	cmdb_ci_compute_template	Virtual Machine Sizes
Cloud Public IP Address	cmdb_ci_cloud_public_ipaddress	Public IP Address
Cloud Mgmt Network Interfaces	cmdb_ci_nic	Network Interfaces
Image	cmdb_ci_os_template	Image
Cloud Storage Account	cmdb_ci_cloud_storage_account	Storage Accounts
Cloud Load Balancer	cmdb_ci_cloud_load_balancer	Load Balancers
Cloud LB Public IP Address	cmdb_ci_cloud_lb_ipaddress	Front End IP or Public IP Address
Software	cmdb_ci_spkg
Software Instance	cmdb_software_instance
Software Installation	cmdb_sam_sw_install

[Matt Hausmann]

Please consider database classes for a future release. It would be important to have both any database instances and PaaS databases included in what is discovered.

[theroz]

How is Azure tag information captured? Does this also create any relationships between subscription CI and resource group CI? 

[christianpreiss]

Hi there, 
thanks for the summary.  

Just to understand it properly the whole ServiceNow portion (e.g. leveraging IntegrationHubETL) needs to be build, isn't it ?

Thanks in Advance !

[Sri104]

Hello, 

I'm looking in to the servicenow store and cannot find it. 

can you please help me in finding it. 

[naxis alx]

similar article for Intune / Jamf is needed. Thanks.

[Robert Wijnbelt]

The Servicegraph Connector for Azure is scheduled for release in mid December.

[vaibhavbhatnaga]

SG-Azure will be GA in Dec

[Will Hallam]

I notice that Windows server VMs are coming in as class "Server" instead of "Windows Server" -- is that expected?

[Stephan2]

Hi. Would it be able to test this integration?

[akata72]

Do you have some more details on why you would require to have Contributor access on the Subscription level for this setup. (i.e what will the App use this for potentially). This might not be feasible in larger corporate environments. 

[akata72]

What API calls will this connector perform?  We would like to test these queries manually. 

[Tim Davis]

Why are servers coming in the base cmdb_ci_server table and not the respective Windows and Linux classes?

[vaibhavbhatnaga]

@Tim Davis , we are classifying the servers to WIN and LINUX in 1.3 version

[Tim Davis]

@vaibhavbhatnaga Do you know when this 1.3 version will be released?

[Stephan2]

@vaibhavbhatnaga do you have a timeline for us?

[Shub1]

@Tim Davis Azure 1.3 will be released on 2023-03-22.

[Aleksey4]

How does it co-exist with cloud resources and CI discovery, as well as Azure alert driven discovery?

do they complement or replace each other, and what's happening if there is a conflict, like issue with the servers coming to the wrong table, described above?

[Denisa Mary]

@vaibhavbhatnaga Do we have any KB articles to add new attributes eg : I want to add IP address, CPUs, Disks , Disks size (GB), Memory (MB) and Network adapters to the fields updated by OOB Robust transform in cmdb_ci_vm_instance table. 

[Marek Meres]

Hi,

If I have ITOM visibility subscription can i download it for free from the Store or need to pay subscription fee for the SGC separately?

thank you!

[Pranav Patil]

Hi @vaibhavbhatnaga,

Can you please share the following : 
1) What is the Oauth time for the token exchange (This is required to share with the security team) ?

2) Is there some configuration be done to rotate the credentials (Like SG connector AWS)?

Regards,
Pranav Patil 

[Aarti6]

@vaibhavbhatnaga We do not have Local Admin account on Production. We will be creating an AD service Account to run the integration. This user will be used in "Run As" in all Azure schedules. 

Can you please confirm what is the minimum role needed to Run SGC- Azure? 

I have gone through the docs and could only find this information for Run as field: - Option to run the scheduled job with the credentials of the specified user.

Please confirm the min role to run this integration.

[Pranav Patil]

@vaibhavbhatnaga 

In Step2 it says : Created workspace for each region under each subscription and noted down the workspace id 

As per discussion with Azure team it is understood that creation of workspace under EACH region is not feasible. Can you please share where this is required to be created ?

Regards,
Pranav Patil 

[Johannes]

Hi,

Will this SG work with Azure Stack?

@vaibhavbhatnaga  @Shub1

[Carsten Schnor1]

Thx for the information. When trying to analyse feature parity between Azure, GCP and AWS I missed a hint about how to collect (in line with ServiceNow strategy) running processes/connected ports (netstat).

Technically for Azure monitoring enabled log analytics web spaces VMconnections would allow for filling cmdb_running_process or cmdb_tcp.

Is there any guidance available how such data is supposed to be collected in line with the product strategy of ServiceNow?

[marcguegueniat]

Hi,

I would like to make explicit that Automation Account and Log Analytics are only required for the software information.

If you just want a resource discovery, you do NOT need them, even if the schema says "Need to setup in Azure"
ie you can just skip steps 2, 3 and 4

Regards,

[ag15151]

@vaibhavbhatnaga 
@marcguegueniat 

Do y'all know what info is pulled from the hardware import and what info is pulled from the software import? Would like to get a better idea of what specific resources each one is pulling to be able to compare. Did some research, but was not able to find much 

[Chris A - UK]

Do we HAVE to enable change tracking and inventory as we just need to pull though Ci's for servers and software?  We want to avoid any additional Azure costs if we can. Thanks

 

[curtisschmidt]

My Azure admin and I are not seeing where in the Subscription -> IAM section we can add the application.  The only options for things to add seem to be users and roles, neither of which contain the application we have created.  Has something changed in the Azure interface since this was written that changes how this is supposed to be linked / setup?

[HiroTokyo]

Same here. I cannot see "add application" in Subscription > IAM . 
Please help for the correct procedure. 

[Amar_Be]

Hi,

Since the release of this article, "tracking and inventory on Automation Accounts" has been deprecated. Azure proposes the migration to the famous AMA (Azure Monitoring Agent).

So can we replace the use of an automation account by the Monitoring agent?

Is there a way to use Azure Functions instead of an automation Account?

I can't find any publication related to Azure Monitoring Agent and Service Graph connector.

[Irston Antao]

Facing issues with enabling change tracking as it’s deprecated as well.

[Frank Eck3]

Hi,

does anyone know when we can expect the Xanadu version of the SG for Azure?

Best regards,

Frank

[HiroTokyo]

@Frank Eck3 
Xanadu version is already available as below
https://store.servicenow.com/sn_appstore_store.do#!/store/application/2489807c1772301015cd34a195bdd5...

[Frank Eck3]

@HiroTokyo thanks, before it was just showing me the version till Washington, thats why I was wondering already.

[FYL]

Does anyone know how the lifecycle management of records work for the Azure SGC ?
Description says it will automatically delete records but fall short of explaining how it works.
Is it configurable by class ?
Does it require activation of  CSDM lifecycle and configuration of data manager policies ?
Thanks.
Life cycle management of records in Service Graph Connector for Microsoft Azure

I found the following KB that describes how this works for AWS SGC for EC2 but not much on Azure. 
ServiceGraph Connector for AWS Does Not Retire Cloud Storage and Cloud Database CIs - Support and Tr...

[FYL]

Recently we encountered the following issue when deploying the SGC for Azure

Please review and upvote on idea portal if you are encountering the same issue.

SG-Azure not pulling Software installs from Windows VMs due to KQL query in GET Software Flow Action
View Idea Page - Idea Portal

[NavaChanikyaAtt]

Hi 
We have recently implemented Service Graph Connector for Azure in both the lower instances, I noticed some field values are missing in the cmdb_ci table like (Model, Manufacturer and Support group) where other than the Virtual machines Model and Manufacturer are like most important fields for any CI, and in cmdb_ci table. Model and Support group is like a mandatory fileds.

I noticed that data discrepancy between the Dev and QA environments where the data updated in cmdb_ci table when I checked by group by class, there was count difference in some classes like Windows and Linux servers, by using following same condition in both the instances.

Can anyone help me on this, I appreciate your support and help in advance.

Thanks and regards
Nava

1. Discovery Source - is - SG Azure (for the Service Graph Connector) (AND)

2. Install status - is - In Use

[Nilanjan1]

Does the SG-Azure Import IP address and FQDN, currently none of the servers which are getting populated from SG Azure is giving me this information ? 

[Dean Attewell]

Can you get this to also discover certs, and expiry dates?

[cjpolanco]

On the Azure side, the Azure team is not seeing any logs for the Service Graph Connector. Would anyone know why? the SGC is updating CIs daily on the servicenow side.