Vulnerability Response and The Discovered Items Module

[Chris McDevitt]

The Discovered Items module is a hidden gem that we can all use to enhance Vulnerability Response and potentially your CMDB.  I hope you find this article useful. Any feedback is welcome.

[Voona Rohila]

The article was very helpful Chris 

I would like to add 2 more points. Correct me if am wrong Chris.

• Unclassed Hardware's, Incomplete IP Identified CI's will have Qualys ID,Qualys Host ID Values Mapped. The field mapping for unclassed hardware's can be viewed in Host Import Maps (sn_sec_cmn_src_cmdb_map) table

• sn_sec_cmn.autoPromoteFields property - To avoid matching low-level networking elements, if a matched CI is one of below classes, the parent CI is returned. 

[Chris McDevitt]

"Unclassed Hardware's, Incomplete IP Identified CI's will have Qualys ID,Qualys Host ID Values Mapped. The field mapping for unclassed hardware's can be viewed in Host Import Maps (sn_sec_cmn_src_cmdb_map) table"

Yes AND... all integrations use this table to build a map that builds the JSON used by the CI Look Rules. The JSON will appear in the Source Data field in the DI Record.

"sn_sec_cmn.autoPromoteFields property - To avoid matching low-level networking elements, if a matched CI is one of below classes, the parent CI is returned. "

Yes AND... this should work OOB

[Chris McDevitt]

Just bumping it to the top of the list. Too bad we can not pin articles.

[Chris McDevitt]

Bumping to top of the list....

[Chris McDevitt]

Bump 🙂 

[Ashutosh Munot1]

HI Chris,

Indeed this is a great article. One confirmation which i want is:

Do the CI Lookup rule run again when we do VIT import? I know it refers to Discovered item table for CI mapping on VIT but for new CIs does it run again.

I think yes but your expert opinion will help alot.

Thanks,
Ashutosh

[Chris McDevitt]

Hi Ashutosh,

When the integration runs, it first consults the Source ID field on the Discovered Items module for a match:

When the integration runs, it first consults the Source ID field on the Discovered Items module for a match:

This Source ID field is a unique value typically supplied by the VR scanner. If the integration run gets a match on the Source ID field, it then uses that records Configuration Item for the host/device.
If there is no match on the Source ID field, then the CI Lookup rules are run to look for a match. Whether a CI match is found or not found, the integration creates a record in the Discovered items table with a Source ID that will be consulted during the next run.

If you want to force the CI lookup Rule to run you can:

1. Us the "Reapply CI lookup rules" list action on the DI module:

2. Make a change (or toggle the Reapply field) to the CI lookup rule and click the "Apply Changes" button:

3. Run the "Reconcile Unmatched Discovered Items" job

[Voona Rohila]

Hi Chris 

If any property value is changed (Added new class in  sn_sec_cmn.ignoreCIClass property) then point 1 or 2 is the best way to handle existing data?

[Chris McDevitt]

#1 is handly to test a change or change a few items.

#3 is kind of the best because it puts into a background job so you can keep track of when it started and finished.

[Voona Rohila]

I think #3 works only for unmatched Discovered items.  correct me if I am wrong Chris

[Chris McDevitt]

You are correct! Sorry... I am focused on "Unmatched" items most of the time.

Yes, use #2 if you want the rules to be reapplied to matched and unmatched alike.

[Chris McDevitt]

Ugh.... You know... It was bugging me... so I dug into #1.... and it calls the  "Reconcile Unmatched Discovered Items" job!

  I looked into that job:

"discoveredItemPgr.addEncodedQuery('ci_lookup_rule.ISEMPTY^ORstate=unmatched');"

So.... it only works on unmatched items....

It appears as if #2 is the only one that will reapply all of the rules.

Sorry....this is new and I am still learning how this all works!

[Krishna100]

Chris,

This note is super helpful! I was struggling at one point on why #1 didn't do anything on matched items.

[Marc Halleux]

Hi Chris,

Great Article i really love it as it condensed a lot of concept in one single doc.

I tried to access the link to the KB Article for deleting VR Data and and it did not worked. After checking with support, it seems the link needs to be adapted to https://support.servicenow.com/kb?id=kb_article_view&sys_kb_id=9799a4451b447414f34d33bc1d4bcbdc

Thanks again for that great info !!

BR

Marcus

[Chris McDevitt]

Marc,

Thank you and.....

Some of the KB articles on Support require a HI account to access. Once you log in to Support you can see the article. I just checked it is still there.

🙂

[Marc Halleux]

Hi Chris,

I was logged on to hi, but i got the following message "You do not have sufficient privileges to access this knowledge item".

So i created a hi ticket and this is the answer i got from the support team:

"I would like to inform you that the KB : https://support.servicenow.com/kb?id=kb_article_view&sys_kb_id=74c1fe4adbcc549c6064eeb5ca961956 is outdated that is the reason that you were not able to access it. Please find below the updated version of this article :

How to delete Existing Vulnerability Response Data for Reimport"

So what i suspect is that maybe the link to the version you have provided in your pdf is not accessible by customer, and the link they provided me is the customer version of it ??

Anyway, thank you for that great article 😉

Marcus

[Ben130]

Hi Chris, love the article!

Is it still true that reapplying ci lookup rules of selected discovered items will only work for unmatched items? I realise 9 months has passed since you posted this...

Thanks!!

Ben

[Wojciech Werysz]

Hello Chris,

Very nice introduction to the topic. Did you create the additional extension of the topics mentioned in the pdf or have more sources how to handle the Unmatched Hardware? Documentation is not very helpful.

[Chris McDevitt]

Hi,

I did write some on Incomplete IP Addresses:

https://www.servicenow.com/community/secops-forum/white-paper-incomplete-ip-identified-devices-and-w...

[Rick Smith]

Chris 

Incredible document thank you!!!

Rick Smith CISSP, CISM, CCNA, CSA

Principal Platform Architect, AMS Federal

(m) 703-282-5099

Rick2.Smith@servicenow.com

www.servicenow.com

[Maloy Banerjee1]

Hi Chris,

Thank you so much for the article. I have one question.

Is it possible to have multiple discovered items 1 Configuration Item? I have 1 CI and for that CI, 27 Discovered Items have been created. Out of those 27, only 1 has VITs, and the rest others are empty. What could be the reason for these many Discovered Items?

Regards,

Maloy Banerjee

[Chris McDevitt]

@Maloy Banerjee1 

Hi, sorry for the late response... a super busy couple of weeks.

Yes, having multiple DI points to a single CI is possible. In most cases, this is a good thing. One example is if a device is scanned in various ways. For instance, if a device is scanned by an "unauthenticated scan" and then by an "authenticated scan" from the same scanning tool, then you want SN VR to associate those events with a single CI.

In the DI module, locate the Source ID. There should be a DI entry for each unique Source ID. You will likely see a difference if you closely examine the Source Data field.

27 does seem like a bit much..... you will need to investigate further.

Tips:
Turn off the UI Policy: "Hide fields for matched or reclassified CIs" to see the Source Data field on "Matched DI."
If you are allowed to use "SN Utils" (a Chrome plugin) in your environment, after Grouping on the CI, you sort by (Up/down arrows next to column name) the grouped count.

[Sangeetha6]

Hi @Chris McDevitt 
Could you please share which script include code create Discovered item from Qualys Host Detection Integreation?

Thanks,
Sangeetha

[Chris McDevitt]

Hi,

I believe the capability comes from this:

[bpolo]

@Chris McDevitt  Thanks very much for the great documentation you provided! I do have a question. Currently the risk_score attribute in the source_data field on Discovered Items  is not being updated when new values are returned from the scanners (we are integrating with Rapid7). We see the latest values in the payloads from the scanners. We are needing to access these latest risk_scores, either my modifying the scripts to update the attribute in the source_data field, or by updating the "asset_risk_score" field on the Discovered Item table(currently it does not seem to be populating). Please would you be able to let us know where in the scripts we could accomplish this? Thanks!  

[Chris McDevitt]

Hi,

I am pretty sure that the risk score for any scanners is brought in as part of the Detection and is also reflected on to the VIT. 

I think that you might want to check out: sn_vul.show_last_open_detection system property.

But... there are no free lunches in life. This adds a lot of overhead and performance penalty to your entire system. 

You may want to consider looking into RaptorDB before enabling that property, especially if you have a large data set.

https://www.servicenow.com/products/raptordb.html