Azure "New Metric Alerts" as an Event Source

[J0pesh]

Hi. 

We're currently in Kingston but about to upgrade to Madrid. So i'll ask this question from a Madrid perspective.

Our Azure Dev Ops team is helping me implement Web Hooks from Azure to ServiceNow, We set a few up already for the "classic Metric Alerts" but they're working to make these redundant and will in the next few months be basing all monitoring within Azure on the "New Metric Alerts" in Azure.

In the ServiceNow documentation, there isn't much (anything) that I can find about new metric alerts.

Anyone got any experience with setting these up and steps I should take to getting it right? I'm still relatively new to ServiceNow and by no means an expert in Event management.. This is a new implementation for us so please be gentle with me 🙂

Thanks

Joe

[dbehnood · Accepted solution]

Hey Joe,

I posted this on a prior thread, but might be useful for you.

The out of the box integration provided by SNOW, documented here (https://docs.servicenow.com/bundle/newyork-it-operations-management/page/product/event-management/ta...), only works for 'Classic' format alerts which are being deprecated by Microsoft on Aug 31st.

The only options available to you are:

1) Create a custom Event Listener Transform Script that can intake Events in the new formats (metric and log being the most common schemas)

2) Send Events to the GenericJSON listener and use a combination of Event Transform and Event Field Mapping rules (https://<<INSTANCE>>/api/global/em/inbound_event?source=genericJson)

I went with option 2 for expediency, and it is working well for us. If you go this route you will need to disable authentication required for the GenericJSON listener (Transform Generic Events MID).

In case anyone is curious, here is how I built this out.

1) Create an Action Group in Azure monitor that calls a webhook action. Specify the webhook URI as https://<<INSTANCE>>/api/global/em/inbound_event?source=genericJson

2) Associate the Action with an Alert rule, and force it to fire (just to capture an Event in ServiceNow)

3) Create a new Event rule:

- Source = GenericJson

- Event filter should include the following condition: schemaID is AzureMonitorMetricAlert

- Transform and compose the following:

a) Description:

Resource: ${data_context_resourceName}
Resource Group: ${data_context_resourceGroupName}

Alert Name: ${data_context_name}
Metric Name: ${metricName}
Operator: ${operator}
Threshold: ${threshold}
Current Value: ${metricValue}

Portal link: ${data_context_portalLink}

${data_context_description}

b) Node: ${data_context_resourceName}

c) Type: ${data_context_name}

d) Resource: ${data_context_resourceType}

e) Message key: ${data_context_resourceName}_${data_context_name}_${data_context_resourceType}

f) Severity: ${severity}

g) Metric Name: ${metricName}

h) Source instance: ${data_context_resourceGroupName}

i) Source: Microsoft Azure

4) We must use regex within the Event Rule rule to get the following nested objects from within the “data_context_condition_allOf” value and convert them into usable values of their own:

• Metric Name (metricName)
• Operator (operator)
• Threshold (threshold)
• Metric Value (metricValue)

The expression used to accomplish this is: .*metricName=(.*), metricN.*operator=(.*), threshold=(.*), time.*metricValue=(.*)}.*

5) Create an Event Field Mapping Rule for Severity (THIS IS A MUST or your Sev 0 Alerts from Azure will force auto-closure of the Alert instead of being most critical)

From field: data_context_severity

To field: Severity

Event Mapping Pairs:

0 -> 1

1 -> 2

2 -> 3

3 -> 4

4 -> 5

6) Create an Event Field Mapping Rule for Resolution State (so that the Alerts auto-close)

From field: data_status

To field: resolution_state

Event Mapping Pairs:

Activated -> New

Deactivated -> Closing

I might have gone overkill on the content I added to the description, but our Ops teams wanted as much info in their Incident as possible. You can tweak this to your organization's needs. Hope it helps!

-Dom

[SNowUser11]

Hi Dom,

Really helpful we are trying to acheive event management from Azure. What I do not know and confirmed is that do we need any additional subscription or license or connector for Azure events. We already have discovery and event management in Madrid now.

Please suggest.

Thanks

[J0pesh]

Hi,

I'm currently working to set Azure monitoring too. I'm no expert, but the way we've set it up is we've created a web hook in ServiceNow and exposed this to Azure.

Our azure instance is using the new metric alerting rather than the classic, meaning we had to use the generic JSON transform script with some tweaks to take the azure metric data being pushed through the web hook into ServiceNow, parse the JSON and push it into the event tables.

As long as you've got ITOM licenses for event management, to my knowledge there are no extra subscriptions needed... 

To use the method in this article to bind the events from Azure to a CI, then you will need to make sure you're parsing out the resource name from the event data. and ideally have cloud discovery turned on into Azure so discovery can access your azure subscriptions and discover resources, resource groups etc.

We found most of the info we needed in existing community articles around how to do this...

[Asela_S]

Hi Dom,

On point 4 above, where do we have to set the New Regex expressions? and how do we "disable authentication required for the GenericJSON listener ". on a side note we are Using ServiceNow Newyork. Thanks in Advance!