What options exist for controlling event floods in ServiceNow Event Management?

[Manas Kaser]

Hello All, 

Hope you all doing well,

This is the first time i`m interacting to a community to seek guidance on one of the task i`m working and the task is totally new to me as  . 

I have to look for an option to control Event Flood in Service Now Event Management. The issue is that towards Incident creation.  I request you to help me on the below issue.

The Monitoring tool is generating Events and alerts which is than getting correlated to  ServiceNow and gets converted to Incident. Now the thing is that the events and alerts keep getting generated however the incident creation don`t stop also, which leads to incident flood. Request you all to guide me with your valuable suggestion on how can i fix this and isolate the duplicate incident creation and allow Event and Alerts to be created. 

Looking for your valuable suggestion/guidence

[robertgeen]

Manas,

What you are describing here sounds like your message_key isn't setup correctly on the alerts. Event floods shouldn't affect the platform that much and what you should see is thousands of events -> handful of alerts -> a couple incidents depedning on your criteria. Not everything should be turned into an incident unless you aren't monitoring the alert console at all. How many are you seeing here?

[Manas Kaser]

Hello Robert, 

Thank you for the suggestion. As for now i `m not able to help on how many are we seeing,  However its been said that multiple events of the same kind occurring in a short period of time.  as Might be the event have unique message key

for now i have this information, I will try to dig more on this to you can guide me on this.

[ori2]

Hi Manas,

In addition to what Robert said, to control the Alert generation from events, you can use Event Rules, to decide which Event will create an Alert and which should be connected to the same Alert, regarding the Alerts to incidents, I assume you are using an Alert Rule which create an incident from an Alert, if not, this is the recommended flow to create an incident, by defining which for which Alert attributes you will actually create an Incident, so configure Alert Rules on top of the Alerts (if you are in London version, so I recommend using the Alert Management Rules).

To summarize:

Flow from Event to Alerts -> use Event rules

Flow from Alerts to incident -> use Alert rules (since London version- Alert Management rules)

Btw, as Robert mentioned, the identifier for the Alerts from the Events is the Message Key, so if you want some events to be linked to the same Alert they should have the same message key (could be accomplished by using Event Rules)

Regards,

Ori

[robertgeen]

Yep the only thing I would add to this is that you don't need an event rule to have something translate to an alert. The only real requirement is to have a severity set (they did it that way so SNMP traps wouldn't automatically convert). Either way ultimately you should look at the event rules you have and the message key on your alerts and make sure that the key is being set properly.