[Mar 2023] Azure Event Driven Discovery with Azure Resource Graph API query

[Ram Devanathan1]

With the March 2023 release of Discovery and Service Mapping Patterns store app, a new mechanism is introduced for event driven discovery for Azure cloud.

The earlier approach for Azure event driven discovery was built many years ago relying solely on the Azure activity logs. Azure activity log forwarding could be setup only per subscription. While this approach did function well for small test / dev environments there were issues with high scale and setting up across many subscriptions, as customers start to use more subscriptions (cloud service accounts records in ServiceNow lingo).

Here's a quick overview of the improvements over older approach -

The azure activity log based approach had several limitations -

• non-secure webhook for forwarding events
• patterns called to process each event coming from azure - more overhead on instance
• activity log forwarding needed to be setup for each subscription
• required additional permissions (beyond reader) to setup activity log forwarding

we were at best doing 3-4 events processed in a minute - this was slowing down the instance when many activity events would come in.

the new approach (aka incremental change enquiry) uses Azure Resource Graph API.

• we are able to get events at large scale across multiple subscriptions at same time
• we are able to process the events faster through the traditional response processing (CAPI) approach
• works for management group also - similar to discovery config
• migrates from old configs towards new config - disables the old configs - check out migration page
• runs in schedule which can be controlled to run in regular intervals

We have tested this with heavy load and see good results up to 27 events processed per second.

Here's the architectural representation of this new mechanism.

Note - currently with the new approach only following resources have out of box response mappings.

• Virtual Machine
• Disk (both OS and Data Disk)
• NIC
• IP
• NSG

Additional response mappings can be created by the discovery admin based on these defaults for immediate needs, we plan to add more in coming releases of the discovery patterns app.

Migration of existing configurations

Existing azure alert configurations can be moved to the new alert processing mechanism there's a page to guide this migration. go to Pattern Designer -> Configure Azure Change Processing and follow the prompts in the page.

This is an irreversible step, migrating to the new approach will mean that - all existing azure alert configurations will be disabled and removed, new alert configuration records cannot be created. However, the benefits from moving to the new approach is tremendous, as it is less intensive on system resources.

Please try this out and let us know your feedback. Thank you all.

Ram

ITOM Product Management

[Krasimir]

Hi @Ram Devanathan1 ,

How about change tracking? With event based discovery the events were stored in the Cloud Events table and it was easy to follow-up the "Cloud Events" either from Activities or from the resource itself on the Cloud User Portal. With the new mechanism - how should that work?

[Ram Devanathan1]

@Krasimir there's capi trails where the information is available - for now that's the best choice.

[Reshma G S1]

Hi @Ram Devanathan1 ,

We had enabled Azure Change processing, from this we are trying to fetch private Ip address from payload and map into VirtualMachineinstance table as IP address is important attributes to connect with Physical servers. As per we analyzed we are not receiving the private Ip address into the payload which is present in sn_cmp_resource_changes_payload_info table for Azure Change process.
Can you help us on Understanding why we are not receiving private Ip address information in payload through the event process(Azure change Processing).

[Jithin1]

@Ram Devanathan1 ,

We have configured Azure change processing for a Management account that has multiple subscriptions under it, is there a way to exclude retrieving events from certain subscriptions or service account under that management account.

We have used "exclude from discovery" attribute to avoid doing scheduled discovery on those Service account.

Is there a way to exclude certain service account from Azure Change Processing?

[Gurmeet Singh2]

Hi @Ram Devanathan1 ,

Did you got anyworkaround to get private IP address of VM in azure

[Jithin1]

@Ram Devanathan1 ,

Is there a way to gather tags of the CIs getting updated ?

[FYL]

@Ram Devanathan1 
We have 2 separate instances of Azure. 
We are currently using Azure event based discovery (Old method - Instance 1). 
We would like to use the Azure Change Processing method for Instance 2.
Are there any concerns to run both methods concurrently ? 
How does Azure Change Processing know which MID Server to use ? Can we specify specific MID Servers as we have segregated MID Servers for each Azure instance.
Thanks !

[akumar130]

@Ram Devanathan1 : Thanks Ram - your post has helped us tremendously in creating azure alert driven discovery.  We are getting the logs in the CAPI table - however looking at the payload we are not able to get the names of the VMs, instead we are seeing the resource IDs. is there a way to fix this ? - Thanks.