Why is Alert CI binding randomly selecting a DNS entry instead of the desired File System or Server entry?

[Katherine Lewis]

The Node contains an FQDN, and all data formats/values are consistent in the event payload.

The event rule will bind to the desired File System CI record with fallback to the Server CI record, but will sometimes bind to the DNS record instead..

The content and format of the desired Server CI records is also consistent.

Also, to answer/preempt some questions:
 - version is Kingston Patch 4
 - and I can confirm we don't have duplicate CI records

Given we have servers that were created by discovery with the following records:

Server A
- Windows server [cmdb_ci_win_server]
 - Name [name] is server-a
 - FQDN [fqdn] is server-a.domain.net
 - SysId is eb9a49cedb225b08d153745bbf96195c

- File System [cmdb_ci_file_system]
 - Name [name] is C:\
 - Mount point [mount_point] is C:\
 - Computer is server-a
 - SysId is 25fc2477dbcfdfc44eb473e9bf961997

- VMware VM Instance [cmdb_ci_vmware_instance]
 - Name [name] is SERVER-A
 - FQDN [fqdn] is empty
 - SysId is c5a4dfe9dbb52200868a7c841f9619ba

- DNS Name [cmdb_ci_dns_name]
 - Name [name] is server-a.domain.net
 - FQDN [fqdn] is server-a.domain.net
 - SysId is 76e05ba9dbb52200868a7c841f961928

Server B
- Windows server [cmdb_ci_win_server]
 - Name [name] is server-b
 - FQDN [fqdn] is server-b.domain.net
 - SysId is 0e926b1dc3100200d8d4bea192d3ae12

- File System [cmdb_ci_file_system]
 - Name [name] is C:\
 - Mount point [mount_point] is C:\
 - Computer is server-b
 - SysId is a72c5181930122005c15d905e67ffbf8

- VMware VM Instance [cmdb_ci_vmware_instance]
 - Name [name] is SERVER-B
 - FQDN [fqdn] is empty
 - SysId is c9edec9c0fc7e200a621fa6ce1050eec

- DNS Name [cmdb_ci_dns_name]
 - Name [name] is server-b.domain.net
 - FQDN [fqdn] is server-b.domain.net
 - SysId is 1dd376fddba21708d153745bbf96199e

And the event rule is configured as follows:

Event Rule Info
- Name is "EventRule01"
 - Source is "EventSource01"
- Order is "100"

And Event Filter
- Ignore is Unchecked
- The following conditions must be met:
 - Type is "EventTypeName01"
 - Metric name is "EventMetricName01"

And Transform and Compose Alert Output
- Configure as follows:
 - Description is ${description}
 - Node is ${node}
 - Type is ${type}
 - Resource is ${resource}
 - Message Key is left blank
 - Severity is ${severity}
 - Metric Name is ${metric_name}
 - Source Instance is ${event_class}
 - Source is ${source}
 - Classification is ${classification}

 - Manual attributes is Checked
  - mount_point = ${resource}\

And Threshold
- Active is Unchecked

And Binding
- Override default binding is Checked
- Binding Type is "CI field matching"
- CI Type is "File System"

When an Event record is created with the following values:
 - Node is "server-a.domain.net"
 - Resource is "c:"
Then the Alert will correctly bind to the File System CI record related to with server-a
 And the Processing Notes are as follows:
  - Binding alert CI process flow:
  - Node is FQDN
  - Node was not found, checking by name
  - Node will be resolved to CI id: eb9a49cedb225b08d153745bbf96195c : found by node name
  - Event CI type is cmdb_ci_file_system
  - Query with fields:
  - mount_point : c:\
  - The event CI type is device, trying to check for matching device
  - Found matching device (using type: cmdb_ci_file_system defined in em_binding_device_map table)
  - Bounding will be done with a matching device (id): 25fc2477dbcfdfc44eb473e9bf961997
  - Bind to 25fc2477dbcfdfc44eb473e9bf961997

  - Event rule applied: EventRule01

When an Event record is created with the following values:
 - Node is "server-b.domain.net"
 - Resource is "c:"
Then the Alert will incorrectly bind to the DNS Name CI record for server-b
 And the Processing Notes are as follows:
  - Binding alert CI process flow:
  - Node is FQDN
  - Event CI type is cmdb_ci_file_system
  - Query with fields:
  - mount_point : c:\
  - The event CI type is device, trying to check for matching device
  - No matching CI found
  - No related CI found for binding, alert CI will be bound to node (id): 1dd376fddba21708d153745bbf96199e
  - Bind to 1dd376fddba21708d153745bbf96199e

  - Event rule applied: EventRule01

[bernyalvarado]

Hi Katherine,

Is it just sometimes? Perhaps the event rule is matching both CIs based on the rule or perhaps the discovered data for a CI is flipping through states that it might make it match sometimes.

Would you mind sharing the event rule that you have configured and if possible... the relevant fields of the DNS and Server CI?

Thanks,

Berny

[stevemacamway]

Katherine - we are facing the same issue. We have not yet determined why it is happening, but think it might be related to the fact that the FQDN has an IP Address that is 'bouncing' around from server to server to DNS, etc.. Depending on what CI the IP address is attached to (in the CMDB) we will get an incorrect CI selected in the Alert. 

In my mind the issue is on the Discovery side, but am trying to understand the process that Event Management uses to map the value in the 'Node' field to CI field. 

What we have on the infrastructure side that I think is causing the problem:

IP Address: 192.168.1.115

Linux Server: server1

DNS Name (FQDN): server1.company.com

DNS Name (application cluster): clustered_application.company.com

Looking back through the Discovery we are able to see that the IP address gets associated with any of the CIs listed above, changing even during the process of a couple related Discovery jobs. 

I'm not sure if that helps, but hopefully it does. If you are having something similar, it would be great if you could respond back and let me know as we'll feel more confident in the direction we are looking. 

Steve

[manivk]

Lewis,

Alert binding to CI  happens using binding rules

https://docs.servicenow.com/bundle/london-it-operations-management/page/product/event-management/reference/r_EMHowAlertsBindCI.html

I am wondering how server CI is having multiple entries in particular class in CMDB . Say for example 'server_A' is linux server and must have one entry in class(cmdb_ci_linux_server) . All DNS names of this server must be in related list  "DNS Names of CI's" of this server CI.

Maybe duplicates of CI is causing issue , remove duplicates and check identification rule which may fix the issue.

Thanks,

Mani

[Katherine Lewis]

Hi All

Sorry for taking so long to reply.

Also, to answer/preempt some questions:
 - version is Kingston Patch 4
 - and I can confirm we don't have duplicate CI records

Given we have servers that were created by discovery with the following records:

Server A
- Windows server [cmdb_ci_win_server]
 - Name [name] is server-a
 - FQDN [fqdn] is server-a.domain.net

- File System [cmdb_ci_file_system]
 - Name [name] is C:\
 - Mount point [mount_point] is C:\
 - Computer is server-a

- VMware VM Instance [cmdb_ci_vmware_instance]
 - Name [name] is SERVER-A
 - FQDN [fqdn] is empty

- DNS Name [cmdb_ci_dns_name]
 - Name [name] is server-a.domain.net
 - FQDN [fqdn] is server-a.domain.net

Server B
- Windows server [cmdb_ci_win_server]
 - Name [name] is server-b
 - FQDN [fqdn] is server-b.domain.net

- File System [cmdb_ci_file_system]
 - Name [name] is C:\
 - Mount point [mount_point] is C:\
 - Computer is server-b

- VMware VM Instance [cmdb_ci_vmware_instance]
 - Name [name] is SERVER-B
 - FQDN [fqdn] is empty

- DNS Name [cmdb_ci_dns_name]
 - Name [name] is server-b.domain.net
 - FQDN [fqdn] is server-b.domain.net

And the event rule is configured as follows:

Event Rule Info
- Name is "EventRule01"
 - Source is "EventSource01"
- Order is "100"

And Event Filter
- Ignore is Unchecked
- The following conditions must be met:
 - Type is "EventTypeName01"
 - Metric name is "EventMetricName01"

And Transform and Compose Alert Output
- Configure as follows:
 - Description is ${description}
 - Node is ${node}
 - Type is ${type}
 - Resource is ${resource}
 - Message Key is left blank
 - Severity is ${severity}
 - Metric Name is ${metric_name}
 - Source Instance is ${event_class}
 - Source is ${source}
 - Classification is ${classification}

 - Manual attributes is Checked
  - mount_point = ${resource}\

And Threshold
- Active is Unchecked

And Binding
- Override default binding is Checked
- Binding Type is "CI field matching"
- CI Type is "File System"

When an Event record is created with the following values:
 - Node is "server-a.domain.net"
 - Resource is "c:"
Then the Alert will correctly bind to the File System CI record related to with server-a.

When an Event record is created with the following values:
 - Node is "server-b.domain.net"
 - Resource is "c:"
Then the Alert will incorrectly bind to the DNS Name CI record for server-b.