Primary alerts shouldn't close the secondary alerts-Event Management.

[gauravbhandari]

Hello All,

I am facing two issues, if anyone can help it very much appreciated:

Issue 1) My first requirement is Primary alert should be converted into Incident and if any secondary alerts comes it shouldn't create an incident. I created a Alert Action rule with condition: Role in Group is not secondary then only create Incident but what i have observed as soon as events converted to alerts, incidents are getting created and Alerts correlation Rules running later to Group the alerts as primary and secondary but at that time Incidents were already created in the system for Secondary alerts.

am i doing wrong something or is it the intended behaviour from service now

Issue 2) Primary alert closing the secondary alerts. How i can changed this Behaviour, where in the system this logic is written. My requirement says Primary alert shouldn't close the Secondary alerts.

Can anyone or Tony Branton please help to understand if this is OOTB behaviour and how i can control or am i missing something somewhere.

Thank you in advance.

[vNick]

You are experiencing intended behavior due to the automated correlation not being an immediate action on incoming alerts (runs every 1 minute).  You would have to build in a time delay to the incident creation flow and have it check the role in group setting after that time delay.

For you 2nd issue, there is no setting for this and the reality is that if it should not be closed by the primary (which should be reflective of the actual issue... somewhat looked at like the root cause issue), then the secondary should be removed from the group because it will also belong to the single incident you want and closing that incident does have an associated property that says to close all related alerts.

[gauravbhandari]

Hello Nick,

Thank you for reply. for 2nd issue we have valid case, let say If network is down which will result in all severs and everything running on network will be down. If network comes up and we can't be sure that everything would be back, there are some cases sever don't comes up and in this case we can't just closed the alerts based on Primary alert.

Is there any custom way to control this behaviour?

Thank You

Gaurav Bhandari

[vNick]

That's certainly a very valid use case.  No out of the box functionality to do that though as it would probably also need to be configured on a use case by use case basis (maybe sometimes you do want the secondaries to be closed automatically).

The best bet would likely be to set the alerts to not close if the incident is closed, and to control any clearing events via event rule so they don't automatically close the primary alert.  Then, you could have alert management rules that facilitate your use cases of closing things appropriately, maybe update the primary if you do get a clearing event (acknowledged and added work notes, etc).

[Christian Ramos]

You can check the script include "EvtMgmtAlertActions" for the delay of running the Alert Action Rules in creating an incident. I would not recommend you to modify this script as it may affect your process when upgrading to London or Madrid.  You can try adding a delay by adding seconds in "evt_mgmt.alert_rule_delay" property so that alerts correlates before creating an incident. By default, it is 5 seconds.

If you are already in London or Madrid, you can create your own subflow in Flow Designer and use it on Alert Management rule that you are working on.