Qualys Integration issues for Vulnerability Response and Configuration Compliance, Host Detection

Lacey L
Tera Expert

‎10-17-2023 06:04 AM

We are running two integration instances in our environment, one for Qualys Vulnerability Response and one for Qualys Configuration Compliance. Below are the jobs running for Qualys VR and Qualys CC. We started seeing duplicate VITs in our production environment when we enabled Qualys Host Detection for CC (the host detection integration had already been running for VR in prod). Should the Qualys Host Detection integration only be running for one integration instance? Is that what is causing the duplicates?

 

Because of the duplicates, we turned off Qualys Host Detection Integration for CC. Now we are seeing empty Discovered Items on the Test Result records. Could this be a result of turning off the Qualys Host Detection Integration for CC or is there a different cause?

 

Below are the active jobs for the CC integration instance (since turning off Qualys Host Detection Integration):

LaceyMorrison_0-1697547335892.png

Below are the active jobs for the VR integration instance:

LaceyMorrison_1-1697547436175.png

 

What is the best practice for jobs and frequencies in both modules?

0 Helpfuls
  • 1,904 Views
18 REPLIES 18

Joe Kline
Kilo Guru
In response to Greg Stone1

‎10-23-2023 12:52 PM

Thanks Greg!  Comparisons are helpful here.  We are at 7 policies, and almost 25M Test Result records.  We, too, put our progress on hold in hopes of some capacity and performance changes by SN.  We estimated adding another 36M records for just one Policy had we enabled it for full production scan use.  Out of curiosity, can you share what page sizes were tried and where you were able to settle on what allows yours to perform better?

 

My experience with the Auto-close seems to be working rather well, within its own other limitations of what data is changed or not and dashboard widget filtering.  I have three rules at the moment based on "last scan" aging at 45 days (not using the last seen, but dot-walked the discovered items.last scan or discovered items.last configuration compliance scan date and as I type this we only have about 800 records that currently meet those filter conditions and should be closed on tonight's run), as well as one where a single CID was removed from Policy for two specific Technologies and one with 16 CIDs that were pulled out of a given Policy.  Since those changes are made in Qualys, there is no way to scan them again.  Since the devices are still being scanned for other controls, the 45-day last scan filter will never close these items out.  Both of those are currently showing no active CTR records.

My issue with the auto-close is that we moved the Integration jobs schedule around a little and ended up preventing the Auto-Close from starting because Qualys PC Results was still running when the Auto-Close tried to launch.

Greg Stone1
Tera Contributor
In response to Joe Kline

‎10-24-2023 08:28 AM

We currently have page size set to 100. We are also looking at lowering it to 75 so we can bring in more policies.

0 Helpfuls

Joe Kline
Kilo Guru
In response to Joe Kline

‎10-24-2023 09:44 AM

I am going to retract a little bit on the success of using the Discovered Item's last scan or last configuration compliance scan dates within my Auto-Close rules.  I just found examples where I have a number of records that are not closed but meet that criteria, and when looking at the Last Seen and record update times, it appears that I am not getting consistent data updates from the integration to update the Discovered item fields matching the results fields.

Back to the drawing board of understanding how SN is storing all this volume of data.

Greg Stone1
Tera Contributor
In response to Joe Kline

‎10-24-2023 11:17 AM

Joe,

Agreed on that one, I'm having difficulties here as well. Another related problem I have is that we were auto-closing using the Result is Failed condition as well. I think this might be skewing some the % calculations we do. We're tracking (failed count) / (total count) using the state field on the CTRs. Let me know what you find out please, I think there may be a fault in the logic somewhere.

0 Helpfuls

andy_ojha
ServiceNow Employee
ServiceNow Employee
In response to Greg Stone1

‎10-24-2023 12:33 PM - edited ‎10-24-2023 12:38 PM

Interesting conversation going here @Joe Kline  @Greg Stone1 

Out of curiosity - are you guys using the Qualys CC integration jobs that use the Qualys PC Posture API (the original API targets) or the newer Qualys PCRS API endpoint?

We used to have an issue back in the day with the Qualys PC Posture API endpoint, in that we would not grab Failed Results if the Status did not change (in other words - we would not true up existing Test Results in CC, so we couldn't tell if a given FAILED record was recently scanned again and eval to FAILED) - this presented a dilemma with shaping an Auto-Close/Stale Closure model -- but has been addressed with the Qualys PC Comprehensive Job.  There is also the newer Qualys PCRS integration jobs which appears to not have this wrinkle...
- https://docs.servicenow.com/bundle/utah-security-management/page/product/secops-integration-cc/qualy...
- This uses the Qualys `lastEvaluationDate` as the time filter for deltas (rather than statusChangedSince)
- https://qualysguard.qg2.apps.qualys.com/qwebhelp/fo_portal/api_doc/pc/get_posture_info.htm

----------------------------------

Here is an old thread from back in the day for reference (no longer an issue with the newer Qualys PC Comprehensive Job that runs weekly OR the PCRS integration):
- https://www.servicenow.com/community/secops-forum/servicenow-configuration-compliance-not-pulling-al...