Recommended practices for CI Matching success (Customers only: deep-dive webinar)

Eliz Skogquist · ‎01-05-2023

Hello all,

The accurate tuning of CI Matching Rules to link CIs for the additional attributes to Vulnerabilities is critical to get full value from Vulnerability Response automation. It is important to get this right.

Join John Gibbons, Principal Product Success Architect, myself and the ServiceNow VR Product Success team for an in-depth review of CI Matching in the Vulnerability Response solution.

The team will demonstrate how to tune CI Matching Rules and will provide recommendations to reach the highest possible matching rate.

Tentative agenda:

- CI Matching Overview
- CI Matching Rules
- CI Matching Properties
- Host Maps
- CI Matching Tuning

To prepare for this advanced webinar and get the most out of it, I strongly recommend that you review these 2 high level articles before the call:

ServiceNow Vulnerability Response CI Matching Tuning

CI Matching - How to do it right.

Please note that these webinars are reserved for ServiceNow customers and certified partners. Please register with your corporate email address so your eligibility can be confirmed.

Looking forward to having you on the call.

Regards,

Elizabeth Skogquist

Sr. Product Success Manager, SecOps

Here is the recording of the webinar for your convenience:

Resource Links

ServiceNow Documentation

*ServiceNow Support - *Must login to Support.servicenow.com to view

You Tube Videos

Here is the full Q&A of the session:

Question	Answer
Where are IRE rules configured where they interact with the CIs created from a VR integration and how can we prevent them from superseding our CI lookup rules?	IRE should always follow the CI Lookup rules. The configuration of matching in the IRE is conducted with Identification Rules. Based on IRE finding a match a CI is either updated or created. If creation is required it is put in the Unclassed Hardware or Incomplete IP table. Check: Identification Rules https://docs.servicenow.com/bundle/tokyo-servicenow-platform/page/product/configuration-management/c...
If there is more than one match for a discovered item, there is field called Other CIs on Discovered Item table. How we can match with one of the CI in Other matched CI field manually knowing the current match is incorrect for any reason?	At this time there is not a way to match with Other CIs on the Discovered Item. You might ssess the source data of the preferred CI and determine if the CI Matching rules order or logic can be updated to match with the preferred CI. If so, make the updates, then select the preferred asset's Discovered Item, then from Actions on selected rows... window select Reapply CI lookup rules to see if the update moves the Discovered Item to be the match of preferred CI.
Tuning CI matching rule is an iterative process, how will you suggest to delete/clean existing records before running next iteration of updated CI ? matching. Any order to be followed for table clean-up	If you want to do a complete reload, you can refer to KB0820838 for detailed instructions on completing a fresh VR load. Check: KB0820838 https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0820838
What would be the recommended setup for environments where the CIs/CMDB is being populated directly by the VR intregration (e.g. SNOW discovery isn't used)?	Recognize that the additional parameters used for automation in VR, like Risk Calculations and Assignment Rules, often uses attributes from the CI. Risk Calculations may use Business Criticality, and Assignment Rules may use Support Group. You will need to reclassify the records into the appropriate classes, they will all initiate in Unclassed Hardware or Incomplete IP>. Then if you can support the process of maturing those other attributes you can ultimately set-up to have data driven risk calculations and assignment.
We are having issues with dynamic ip addresses. users working from home and different places have dynamic ip addresses. we are having problems with multiple devices having the same ip address. how do we handle this?	You may want to update the script in the IP Matching Rule to exclude the Computer class (or according to your organization's class which has the equipment with the dynamic ip address).
There was an earlier question "Is there a way to prevent assets coming in from the scanner payload to from becoming a discovered item in the first place - this is partically important in the cloud where we have ephemeral assets that have already been tagged as such."	Depending on which integration you are using, you can filter on the scanner side and use that filter during load. Otherwise, configure the Auto-close Configuration to close stale or retired CI.
How the duplicates are handled during CI match ? If CI is created in 'unclassified hardware' and later it is discovered by another source but due to weak CI IRE rules it mark correct CI as duplicate of "unclassfied hardware' one. will it continue to match to unclassified one ?	First step in processing is to check the Souce ID to see if the asset has been previously processed. If so, it matches that Discovered Item record and skips further processing of CI Matching Rules or IRE. So in the scenario you describe, it will continue to match the unclassified one.
ok thanks	?
The CI matching process does not match if the “duplication of” configuration item attribute is populated. The duplicate of is widely use. Thoughts on a solution?	If a CI is duplicated in the CMDB when the CI Lookup Rules find a match they will pair with the most recently created CI.
When the CI lookup rules are reapplied to a small number (fewer than 10) discovered items, approximately how long until the vulnerability items are updated? Is this a 30 second process or an hour process, etc.?	Updates on selected Discovered Items having CI Lookup Rules reapplied should happen instantaneous. If you are anticipating fewer than 10 to update when running the Reapply CI Lookup rules using the button(UI Action) from the CI Lookup Rules listing it will process the entire Discovered Item table to find those 10, which could take a few hours, if the table is in the millions.
Something I’ve noticed with Qualys is that the host list integration vs the host list detection do not send the same attributes in the payload. The problem that happens is this can cause CI lookup rules dependent on attributes in the one integration to not fire in the other - we have been able to work around this with a combination of tags and ip address. Other wise a mismatch CI gets created and then that gets matched moving forward. Is this is issue most likely the payload being given by Qualys?	Processing of Host List is building the asset table (Discovered Items) and Host List Detections brings in the findings, matching to the Discovered Item and creating the VIT. The payload coming in from Qualys is going to drive matching and VIT creation.
Qualys generates a different source id for the same CI based on Tracking Method this generates a duplicated Discovered Items, This there something we can tune around that?	Vulnerabilities coming from different scanners currently do not have an OOB way to be recognized to avoid creation of duplicates. A custom script will be needed, however this capability is coming in a future release.
When a CI is not matched, can we disable automated creation of IP Address and Network Adapter CMDB recrods link to that unmachted CI? And if the CI is than matched to a correct CI in the CMDB, how can we get rid of these records (IP Adress and Network adapter) ?	If the system property sn_sec_cmn.ci_creation_through_IRE is set to true, you cannot disable the creation of associated Network adapter and IP records. If you set the property to false, it will then create Unmatched CI class records only, however these CI's will never be reclassified
Can you share the KB related to truncation cascade	Of course, KB0820838 https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0820838
If a Discovered Item is created and it is a bad match, will re-running the CI Lookup rules fix it if the Rules have been updated to provide a better match?	Yes, it reassigns the CI if the Lookup Rules determines an update of CI on the record.
What would be the best approach to fixing an unmatched CI with low to empty source payloads?	Wait to see if future scans of the device bring in additional attributes for the asset. If over time it does not build out with more details you could delete it. Know that the following scan that includes that CI it will rebuild the record.
It is my understanding the the IRE will first try to find a match before creating a duplicate CI, is that correct?	Yes, IRE processes after the CI Matching Rules to find a match to update, and if not found creates an Unclassed Hardware or Incomplete IP record in the CMDB
What would be the best approach to fixing an unmatched CI with low to empty source payloads from the scanner? via the scanner tool or is there a script to look at?	Wait to see if future scans of the device bring in additional attributes for the asset. If over time it does not build out with more details you could delete it. Know that the following scan that includes that CI it will rebuild the record.
Also, is there ever a time where we should use auto-flush rules to flush out data on Discovered Items table (maybe retired CIs, never to match CIs)?	Data hygiene is an important discipline in VR, as tables can get large over time. To maintain optimal performance, you will want to consider Auto-delete or Archiving data.
In auto promote property, is it possible to look for second level and third level CI ? network adapter > VM instance > Windows Server?	Currently the property will only use the CI field associated with the parent record. You may want to post that suggestion to the Idea Portal for consideration of a future enhancement
what are the fields present in CI Lookup Rule condition? Source data?	Depending on the VR Integration plugin installed the values used in the condition: sn_vul_manual_ingestion_host_attrb sn_vul_r7_dwh_host_attrb sn_vul_r7_ivm_host_attrb sn_vul_qualys_host_attrb
Is there anything in a roadmap to allow for configuration adjustments to the unclassed hardware CI? Right now any adjustments to either name or to include OS in the unclassed hardware needs to be done via script	Host Maps will allow you to define what fields are populated in a Unclassed hardware record when created.
Is there a plan to stale out Discovered Items	If the related CI is decommissioned
Is there a place to write custom script to find Parent CI if we don't have CI field on the base table?	This requires logic in the CI Lookup rule. You will need to be careful to avoid multiple or complex queries in CI Lookup rules so as not to affect ingestion performance.
What is the recommended practice that oragnisation can opt to reapply CI lookup rule? is it monthly / quarterly?	Depends on data volume which are unmatched. We have seen customers work with quarterly, but not for all data. (how is "all data" being filtered out?)
In our environment we still see the legacy class matching "Unmatched CI", what does this mean? and how to correct this?	You will want to make sure the system property ‘sn_sec_cmn.ci_creation_through_IRE’ is present and has a value of ‘true’. If that property is true and you still see Unmatched CI Class records being created then something is failing with the IRE Module - you will want to open a HI support ticket to dianose the issue
The Discovered Item table is not opening up in my instance	You will want to make sure the VR Plugin is installed. The table is under the ‘Security Support Common’ application which is a dependency of the Vulnerability Response plugin.
Is the ignoreclass property used by IRE as well?	No - IRE is used only to match a CI for update or create a CI when one does not exist.
Is there a way to prevent assets coming in from the scanner payload to from becoming a discovered item in the first place - this is partically important in the cloud where we have ephemeral assets that have already been tagged as such.	No, there is not a way to do this without a complete custom overhaul of the CI matching/creation process. What is the current challenge you’re having with ephemeral assets?
Is it a good practise to set the unclassed hardware/incomplete Ip to the ignore classes and CI lookup rules never check for this tables as a fallback option?	No, this is likely just result in either duplicate CIs created by IRE, or constantly changing values on those Unclassed/Incomplete CIs. It is an interesting idea though.
is there anything in the roadmap to launch a quick discovery from the unmatched ci records themselves?	Not currently, but very interesting concept Billy. Would you mind submitting this in the Idea portal on Support?
What is that system property to prevent IRE?	sn_sec_cmn.ci_creation_through_IRE
What if I don’t want to ignore a CI Class, only prioritize a server over dns if and when that situation occurs. In this case, ignoring a class is really not an option. Should I just write a script for “cmdb_ci_server” first and then the whole CMDB or is there a better way?	Somewhat - you should write a CI lookup rule that looks for server specifically, and ensure it runs before a lookup rule that would find just anything based on dns
What is the difference b/w source display value and source id in the discovered item record in source data value?	The ‘Source Display Value’ showed in the screen shot was the ‘display name’ of the integration used to bring in the discovered item. Source ID is usually the ‘asset id’ value from the scanner
when is the unmatched ci recorded generated	The CI which is generated is an “Unclassed Hardware” CI. It’s the related Discovered Item record which gets the state “unmatched”, and that is generated after the CI Matching rules DO NOT find a match, using that IRE payload.
I see the duplicated entires of the source Automox and Teanable source is no creating an de-duplicate task, does this mean there is an failure in IRE which i will have to look at before implementing the VR	The De-duplicate task is not created by VR. I am not sure if the Service Graph Connector for Automox uses IRE as part of the ingestion process. I would open a Ssupport ticket to get clarification
table name for discvoered items please	The table name is: sn_sec_cmn_src_ci
there are three types of unmatched records generatered 1) incomplete ip, 2) umatched ci or 3) unclassed hardware - when is the unmatch ci record generated	The Unmatched CI Class record is created when the system property ‘sn_sec_cmn.ci_creation_through_IRE’ is set to false OR if there is a failure with IRE when trying to build a unclassed or IP record. If you are seeing a large volume of unmatched CI Class records being created and your system property is true, you may want to contact support to investigate the issue
The re-running CI lookup rules feature for Discovered item is based on Discovered Item Number? Is ServiceNow looking into enhancing this feature so that we can query bulk DI based on certain conditions for mismatched or unmatched CIs and re-apply the CI lookup rules?	There is just the list action which allows you to select, and the reapply to entire table as just shown. We do not currently have plans to allow reapply by condition.
There's a property which can be set for auto-promoting records to a referenced CI (e.g. VMware Network Adapter > Virtual Machine Instance). How would you go about auto-promote based on CI Relationships (e.g. VMware Network Adapter (reference field) Virtual Machine Instance (relationship Instantiates::Instantiated by) Windows Server)	There is not currently a way to do that, without scripting it into a CI Lookup Rule.
Performance impact on having several rules?	This will depend on if the Rules are Field Matching or Scripts and the efficiency of the scripts. You can see the impact of your CI Lookup processing, if its excessive, by going to the Integration Run job, and on Complete, the Performance Statistics section displays the time used in CI Lookup. Often time that value is seconds
We bring in Assets as part of Qualys Vulnerability Response integration. What parameters from Qualys such as host tags can be used for reconciliation and classification so appropriate class is assigned to CIs	Unfortunately no, in order to change the parameters that are fed IN to IRE for CI Creation, you would need to customize the script that formulates the payload for IRE. While I wouldn’t recommend this as it is a deep customization, it is something I have done before that is possible to do by clever modification of a few lines of script. Look at the script include ImportHostCmn, which will be called by the QualysHostImportReportProcessor as “ImportHost”.
Do you have a recommend starter list for ignoreCIClass and autoPromoteFields system properties?	Unfortunately we don’t have a starter list. If you are working with a Partner, I am aware some of them have them.
thanks Jon, we haveimpplemented tenable as a source . what should be my next steps to enable the scanner , and how do i enable this grouping or CI matching rule	Use the Setup assistant for VR. https://docs.servicenow.com/en-US/bundle/tokyo-security-management/page/product/secops-integration-v...
If a previously unmatched CI is later matched to the correct CI after reapplying CI lookup rules, are the assignment rules rerun on the associated VIT?	Yes
This process of CI lookup and Discovered items is shared logic for Vulenrablility response and configuration complaince right?	Yes - the logic is in Security Support Common scope. See the chat for a listing of the script includes, but start by looking at the ImportHostCmn script include. You'll find the Discovered Item creation function on line 326 “_createSrc”
Will changed CI Lookup rules run again on already imported and matched discovered items that are imported again on a new scan, or do we need to manually reapply the lookup rules?	Yes, you’ll want to reapply against the entire table.

And the PDF version of the slides:

Smitty · ‎01-10-2023

Hello. I'd like to attend this but my registration request was denied for some reason. Could someone provide guidance on how I can be allowed to attend?

Eric Feron · ‎01-19-2023

Hi @Smitty , this is a customers-only event and you need to register with your corporate email address.

Thank you for your understanding.

Cheers,

EF

Eric Feron · ‎01-19-2023

Hello all, only a few days left to register. Don't miss this one. Get your CI Matching right.

See you there,

Cheers,

EF

Eric Feron · ‎01-25-2023

The first session is starting in less than an hour. See you there.