Event Management - MID Server -2 questions - Processing Traps & syslogs.

[dan_tembe]

Hello Team, 

Wondering if someone can shed some light and help me understand the workings of a MID server in relation to processing inbound SNMP traps and Syslogs?

1) Syslog processing - Can a MID server receive syslogs on the standard Syslog Port (UDP-514)? if so is this something I need to enable on the cloud side to push the config to the MID or something on the MID server conf file that I can enable this? 

If yes, is there a way to format syslogs before they reach the em_event table. 

if No, any recommendations/suggestions on syslog to trap or syslog to REST/JSON (open source - like Graylog or ELK) something you are using now to feed data to em_event?

2) SNMP Traps - Inbound to a MID server. Does the MID server, when receiving inbound SNMP traps, utilize the MIBS (esp that define the trap format) to map the varbinds to em_event fields? 

I dont see this working, when I do load the MIBs; but that could be because I don't have the correct dependency mibs loaded. Otherwise, the MIBs might be only for inventory - discovery via MID. 

If MIB's are not used my MID server for inbound SNMP trap formatting natively, is there a different way I can format the trap varbinds from various tools / devices traps -> to format properly before they reach em_event table.

I know I could do this with NNM, Netcool Omnibus.. etc.. but wanted to see if MID could natively do this.

Basically, I am looking for more intelligence in trap handling at the MID server, almost like the work that happens at em_event table, happen on-prem, so the flood of events can be managed using de-dup keys and suppression on premise before the load is put on the cloud SNOW EM instance. 

Thanks in advance for your input. 

Regards, 

Dan

[robertgeen]

Dan,

I'm not sure about the first question but the second question the answer is yes it should translate them. I have had issues with it in the past and it's almost always dependency issues. Make sure dependent MIBs are loaded first and then load the others into ServiceNow. I have still had some issues where only partial varbind have happened but it was enough to get through it. I would check your dependencies and then you should be good.

The syslog is an interesting one and I have never seen anyone doing that nor do I think it's possible but someone else might know better then me. Thanks.

[dan_tembe]

Hello Robert, 

Thanks! I will run through the MIB dependencies and make sure I have them loaded.

Once the MIB is loaded.. as far as mapping the fields, node or type or description field to a varbind is still using a rule or is there a way to do this at the MID server? 

Where I am getting at is, when I scripted integrations into em_event table, I took the fields from a tool event table, that I wanted and mapped to various fields in em_event table, including the message_key, so it made the data show up clean in the em_event table which helped me create clean rules. Whereas with traps, most of the information is populated in the Description field with RAW trap varbinds, and every rule I create need to 1st include mapping the varbinds to proper fields and at the same time manipulate the message_key.

Just was trying to simplify the look and feel at event table view. 

On the Syslog question. I was hoping there was a way to ingest Syslogs via MID since security management relies on syslogs & ServiceNOW has the Event Management integrated with the security module.

Having to use a splunk or other commercial aggregator is OK, but cuts into the ROI and essentially, if we have one place where all events are coming in (em_event) table, then it is easier to group, analyze and create alerts and then security incidents.

Secondly, MID facilitates the On-prem to Cloud communication in a secure / approved way for servicenow so I was hoping syslogs is just something I missed in my review of docs. 

Thanks again for your response, 

Dan

[robertgeen]

Dan,

So when a trap comes into the mid server it actually takes all the trap oids that match and dumps them into the description field which in the end gets dumped into the additional_info table. You will have to match using regular expression in the event rules. Unfortunately because the source could be just about anything it's dumped in in a way that it's expected that you parse it with an event rule. You will see in the additional field 1 to many entries in json for each of the oids with information and if the MIBs are done right they should translate.

Yeah I see what you mean about syslog but I've unfortunately never dealt with them before and I'm not entirely sure ServiceNow can do it. I believe Splunk has an integration for pushing events into em_event but I haven't touched it at all. Other then that I would open up a HI ticket and see if they know of a way to do it. If you are on Kingston it's possible you could use the API mid server call and create a custom transform script to transform them into the data that you want but I suspect that negates the whole purpose of having syslog messages come through syslog in the first place.

[dan_tembe]

Hello Robert, 

Thanks! for this information. I do see the OIDS (sometimes with the leading enterprise matched (I am sure from the MIB in MID Server /agents/work/MIB/ folder) match. I will do the - open a service ticket to ask for some help from EM team. 

For the Syslogs, I could just push them to an existing monitoring tool, then I already have the REST/JSON integration using a groovy script in place, to dump these into em_event, as a last resort. But I need to look at licensing costs for this tool. 

Thinking out loud, it just needs to be a tool / app that takes in Syslog --> inbound on UDP 514 and maps fields in the syslog to ServiceNow event and writes it to em_event over JSON using REST/SOAP. Heck even converting Syslog to SNMP trap will work in this case, if I need to use the MID today. But any custom development means upgrade hassels down the line, so looking for out of the box functionality from the MID. 

I am going to follow up with our internal ServiceNOW team, they have the ability to open case with support.

Thanks!

Dan