ServiceNow Configuration compliance not pulling all data from qualys on OOB API parameters

[IceIronDragon]

Hello All,

We are seeing this weird behavior on ServicewNow Configuration compliance :

>>where in we are using OOB API parameters.

>>test results are getting created 

>>but the amount of failed test results generated by Qualys (weekly qualys report) is 2times more than what we see on ServiceNow .

>>there is no filteration or custom Parameters used.

>>anybody faced this issue ??

[andy_ojha]

Hey there,

Great observation.  This is a somewhat common experience with the native filtering used today with Qualys PC.

The default 'time-based' Qualys API time-based filter parameter used here, when fetching Results from the Qualys PC Posture API - is tied to "Status Changes Since".

How far back did you set the 'Import since' date to for the first initial load (Qualys PC Result - Job)?

This is key, because that will control the scope of 'FAILED' Results pulled into NOW CC.

The current integration does not pull in 'FAILED' Results from Qualys PC, based on "Last evaluated date" -- meaning just because a FAILED Result was seen on a Host yesterday, it does not mean that data comes into ServiceNow.

The mechanism uses "Status Changes Since" -> which is more restrictive and limits the Results brought in from Qualys PC, to those where the Status on the Result has changed since a given date / time .... between: PASSED <-> FAILED <-> ERROR ...

If you are by chance familiar with the NOW Qualys VR integration - the mechanism for pulling in data works much different with VR than it does for CC.  For Qualys VR - detections are pulled based on simply being 'seen again' or updated ... rather than limited to when their Status changes...

-------------------------------------------------------

The other thing to double check is the permissions of the Qualys account being used to fetch data from Qualys PC via the API - if the account is limited in permissions / access it may not be pulling in all expected Results from the given Hosts you have being scanned for compliance.

Reference: - Qualys "Status Changes Since" Illustration

[IceIronDragon]

This is very helpful.

>> we had the data pulling from the day we got the qualys integration done

quick questions :

>>when you say "change in status"  does it mean any update on the CI 

>>because from your diagram it shows let say 100 failed test but only 50 test failed test goes to servicenow ,  why is that ?   

>>we are trying to pull data on daily basis and only few data comes into servicenow

[andy_ojha · Accepted solution]

Hey there,

It's driven by the Qualys PC attributes ...

"Change in status" -- means change in the Qualys PC Result Status -- the filtering here is based on data in Qualys PC not ServiceNow

Let's say on your Qualys PC Result Job - the `Import Since` is set to May 1, 2022 

• This translates to the API request going to Qualys PC as "hey Qualys, give me all compliance posture results, that have had a status change since May 1, 2022..."
  
  
• This will bring in Qualys PC Results (Compliance check for a control on a host) - where the Qualys PC Result Status has changed since May 1, 2022 
• This will exclude Qualys PC results that had their status change prior to May 1, 2022 -- even if they are continuously evaluated every single day from May 1 to May 26 as FAILED over and over again 
• For example, Qualys PC FAILED Results that had their Status initially set or last changed in Jan 2022, Feb 2022, March 2022 -- would not be brought into ServiceNow
• For Qualys PC Results that were in a FAILED Status long before May 1, 2022 - those would not be brought into ServiceNow 
• Even if those same Qualys PC Results are evaluated every day as FAILED over and over again - between May 1 and May 26 - they would not be brought into ServiceNow 
• Change in Qualys PC Status maps to the Qualys PC Result (FAILED <-> PASSED <-> ERROR)

Backdating the 'Import since' on the Qualys PC job for the very first import can be used to initially set the benchmark / pre-seed the data, and from that point forward only delta updates are bought in daily.

After the first initial import (with the back dated 'Import since') -- actionable data will be brought into ServiceNow from that point forward... i.e. Qualys PC results with a status change (FAILED <-> PASSED <-> ERROR)...

Keep in mind - this will also impact how the the 'Last seen' field on ServiceNow CC Test Results is maintained due to the filtering limiting Qualys PC Results based on 'Status changes since'... ('Last seen' won't be updated every time a host is evaluated against a control over and over, unless the Status of the Qualys PC Result changes)...

[Adam Peterson]

Thanks for this explanation. Better than other docs I've seen!

I do have another question about the Qualys integration. If I want to bring in an additional field from Qualys into Service-Now, how do I go about adding that? 

We are trying to pull in netbiosName from Qualys. 

Thanks for your help!

-Adam