Qualys Integration issues for Vulnerability Response and Configuration Compliance, Host Detection

[Lacey L]

We are running two integration instances in our environment, one for Qualys Vulnerability Response and one for Qualys Configuration Compliance. Below are the jobs running for Qualys VR and Qualys CC. We started seeing duplicate VITs in our production environment when we enabled Qualys Host Detection for CC (the host detection integration had already been running for VR in prod). Should the Qualys Host Detection integration only be running for one integration instance? Is that what is causing the duplicates?

Because of the duplicates, we turned off Qualys Host Detection Integration for CC. Now we are seeing empty Discovered Items on the Test Result records. Could this be a result of turning off the Qualys Host Detection Integration for CC or is there a different cause?

Below are the active jobs for the CC integration instance (since turning off Qualys Host Detection Integration):

Below are the active jobs for the VR integration instance:

What is the best practice for jobs and frequencies in both modules?

[Greg Stone1]

We are using the PCRS integration - and I am looking into what appear to perhaps be some issues. Our auto-close logic was set to be 10 days for the failed test results. I'm having that failed test results condition removed. I'm still seeing cases where CRGs are closing when there are open CTRs, and I think it might be because the CTRs are not getting stale closed.

[Joe Kline]

Hi Andy - thanks for the curious addition to this conversation

In my case, "Yes", I am still using the Qualys PC XXX set of Integration jobs with Results using the Posture API.  I have wanted to attempt to trade off over to the new PCRS version since it popped up in one of our previous version upgrades, but my team partners kept putting it off and a recent re-org had my new management put the whole series of investigating some integration issues "On Hold".

We have the similar issue you describe (I think) in that since we only pick up changes in status, we are not fully "true" on millions of test result records.  We thought we would be able to make use the Discovered Item Last scan fields, but I found out that is NOT happening correctly either - and I have some CTRs that show Last Found of yesterday and last updated of today while the referenced SDI says last scanned in May.  I also have an ongoing issue that my Results integration tends to Fail daily with just one to three of the VINTPRCs ending in a "Job exceeded processing time limit" error.

We just updated to latest versions again mid-last week, and now have the Comprehensive integration so I can see if that helps us out similar to your description, and what I may or may not need to do for my auto-close/stale conditions as well.  My first production run of that will be this coming weekend.

On the PCRS front, I am trying to re-energize that concept with my new management to lift the on-hold if it means we can get better trust in what does come over, even though we have put our Qualys PC rollout on hold because we will quickly (if not considered already having done it) blow the doors off SN storage and performance with the load of records CC has with it.  If I get that, I am still wondering a bit about the data storage and which fields are correctly to be used in reporting and things like auto-close when (for example) using last evaluate to filter the data, but last evaluate is not retained in the data tables once over in SN ... I have had several items appear where I challenged SN storage of data and some they agree with (created a fix for me, but then said they were not sure they agreed enough to make it a fix for the product for everybody), some they agreed and turned my case into a feature request.

Your feedback makes a second one that really points to PCRS as being a problem solver that I need to attempt to get to, it seems.

Thanks, again, for this dialog opp!

[andy_ojha]

Hey Joe - roger that...

Yeah the Discovered Item bit is interesting - there might be an explanation for that - but would take some time to diagnose ... (probably worth a Support Case to review)...

Given you have some trouble with the max processing time - may need to look at the pagination for the jobs (especially with that Comprehensive job).   I think baseline it is around 5000, could reduce this down to 3000 or 2000 and see if we get the files processing more quick (under an hour).   It will result in more files (less chunky) - but increases the chance of them being processed in under 1hr which is that limit you bumped into.

If we see issues with the regular Results job - would suspect for sure we will see this with the Comprehensive job too..

Nav to Primary Integrations -> Open the Qualys Results job (the regular PC one or Comprehensive) -> Qualys REST Details Tab -> REST Method -> open the record (Name = List)

On the Query Parameters -> try reducing down to 3000, 2000, etc...

Then - if you get the Comprehensive job working smoothly - you could look at using something like the Test Result (Last Seen) date .... rather than the Asset Last Scan from the Disc Item...

That might hold you over until migrating to the PCRS API.

[Joe Kline]

Appreciate that perspective, Andy!  My truncation limit is set at 900 for the PC Results integration, and it appears that perhaps the Comprehensive uses the same REST Details method because it too is showing the same 900 value.

[Greg Stone1]

For the auto-close - we were set up the same as you - discovered item.last compliance scan date, but again, I'm seeing potential issues with that. I'm thinking since we both had the same date in there Service NOW must have recommended we use this field for some reason. Personally, I'm wondering if Last Seen Date would be better.