Whitelisting observables in ServiceNow

Filip Vojt__ek
Mega Guru

3 hours ago

What it can bring you

 

  • Reducing the noise and false positive which need to be reviewed by Analyst and thus reducing MTTR (Mean Time To Resolve)
  • Better security - by not sending out information about your internal infrastructure
  • Performance of the platform - by not running unnecessary flows and logic
  • COSTS - some integrations are paid by the number of requests you make to the API, whitelisting can significantly reduce the amount of calls

 

Safe harbor notice

Keep in mind that this implementation is just a showcase to give you an idea. A lot of things are done in sake of simplicity and keep the scope on the topic (ie. accessibility of Script Include, UI Actions conditions etc).

This is not a final product! Goal is to give idea how to configure.

I'm not taking any responsibility for use of the practices shown in this article.

What you would need

 

  • ServiceNow PDI on Xanadu or higher
  • Security Incident Response [sn_si] plugin installed
  • For ease of testing and access you might also want to install Threat Intelligence [sn_ti_ac] module
  • Security Incident Response admin [sn_si.admin] role and Threat Intelligence admin [sn_ti.admin] role
  • FOR TESTING If you don't have any integration for observable enrichment you would need MISP integration for Security Operations [sn_sec_misp] (no need of acutally having MISP instance)

 

Recognizing Observables to be whitelisted

First of all we need to think about based on what we will recognize whether the observable should be whitelisted.

The observable value for whitelisting would probably fall into one of these categories

 

  • Hardcoded values
  • IP Ranges
  • REGEXes
  • More complex rules (ie. Observables matching Windows Server CMDB records)

 

The Approach

The idea is to use as much OOB logic as possible. Luckily enough we already have a handy tool to for this - Security Tags and Security Tag Rules.

First, switch your Application scope to 'Security Support Common' and create an update set.

Now, since we are all set, let's

 

  1. Find a suitable Security Tag
  2. Define rules for that tag for all categories (ie. IP Ranges, Regexes).
  3. Excluding whitelisted Observables from integration capabilities
  4. Roll-up the rules against observable table (this will be covered in part 2)

 

1. Find a suitable Security Tag

 

  1. Go to the Security Tag [sn_sec_cmn_security_tag] table to see the whole "menu"
  2. Choose one, I strongly suggest taking either 'Enrichment: Allowlist' [71562f6fc7d310101917dac2c7c26036] or 'Enrichment: Whitelist' [e6e6ea180b503200263a089b37673a08]. We will use 'Enrichment: Whitelist'.
  3. If you open the 'Enrichment: Whitelist' record, you may notice that Security Tag Group is empty. Either you can select 'Enrichment allow / denylist' (which we will do for sake of simplicity) or create your own. Save the record (this will make it available on the Security Tag selection).

 

2. Define rules for that tag for all categories

A) Hardcoded Values

This is as simple as it gets. We basically need to create a condition for each value. Based on needs of your company, you can create one and place all conditions there, or divide it by observable type or any other parameter you fancy.

You should keep in mind the maintanance (whether it will be done by developers or handed over to Analyst teams [preferable])

 

  1. Go to the 'Enrichment: Whitelist' record on Security Tag [sn_sec_cmn_security_tag] table
  2. In Related list 'Security Tag Rules' click on 'New'
  3. Fill in the 'Name' (ie 'Hardcoded observable values'), keep in mind your company naming convention
  4. Make sure 'Active' is ticked true
  5. For table select 'Observable' [sn_ti_observable]
  6. In Condtion field select 'Value' and then define the condition based on what you want to whitelist (ie. IS 'google.com'). Note that you can either use OR condition or create a new rules, depends on the grouping you are looking for.
    1752160508489.png
  7. Submit the record
  8. For testing go to the Observable table [sn_ti_observable] list
  9. Click on 'New'
  10. Fill in the Value 'google.com'
  11. Submit the record
  12. Open the record you have just created
  13. See that it has tag 'Enrichment: Whitelist' attached to it. Also, if you open the XML of the record, you can see the sys_id of the tag in <security_tags>. 
    1752160862448.png
  14. Notice that also if you go to the 'Enrichment: Whitelist' record on Security Tag [sn_sec_cmn_security_tag] table, in related list 'Applied Security Tags' you can see the M2M record
 

B) IP Ranges

 

  1. Go to the 'Enrichment: Whitelist' record on Security Tag [sn_sec_cmn_security_tag] table
  2. In Related list 'Security Tag Rules' click on 'New'
  3. Fill in the 'Name' (ie 'IP Range 172.16/12'), keep in mind your company naming convention
  4. Make sure 'Active' is ticked true
  5. Tick 'Use filter group'
  6. 'Filter Group' field appeared and you can select one of default filter groups, for simplicity we will go with "Local IP Address Range '172.16/12'". Here you can also create a new one, featuring whatever IP Range you fancy. Just change the 'Network IP Address' and the name and use 'Insert and stay'. 😉
    1752161597682.png
  7. Submit the record
  8. For testing go to the Observable table [sn_ti_observable] list
  9. Click on 'New'
  10. Fill in the Value '172.16.20.30'
  11. Submit the record
  12. Open the record you have just created
  13. See that it has tag 'Enrichment: Whitelist' attached to it. Also, if you open the XML of the record, you can see the sys_id of the tag in <security_tags>. Note that 'Block From Sharing' tag might be also added, since that is OOB configuration.
  14. Notice that also if you go to the 'Enrichment: Whitelist' record on Security Tag [sn_sec_cmn_security_tag] table, in related list 'Applied Security Tags' you can see the M2M record
 

C) REGEXes

Since there was an issue with ServiceNow 'matches regex' operator described PRB662349, which is no longer availabe I would strongly suggest to do a thorough testing and if you experience any issues, just solve it with D) More Complex Logic.

 

  1. Go to the 'Enrichment: Whitelist' record on Security Tag [sn_sec_cmn_security_tag] table
  2. In Related list 'Security Tag Rules' click on 'New'
  3. Fill in the 'Name' (ie 'Single digit'), keep in mind your company naming convention
  4. Make sure 'Active' is ticked true
  5. For table select 'Observable' [sn_ti_observable]
  6. In Condtion field select 'Value' and operator 'matches regex', then the regex in form '^\d{1}$' (without quotes, for single digit)
    1752182206691.png
  7. Submit the record
  8. For testing go to the Observable table [sn_ti_observable] list
  9. Click on 'New'
  10. Fill in the Value with any single digit (ie '7')
  11. Submit the record
  12. Open the record you have just created
  13. See that it has tag 'Enrichment: Whitelist' attached to it. Also, if you open the XML of the record, you can see the sys_id of the tag in <security_tags>.
  14. Notice that also if you go to the 'Enrichment: Whitelist' record on Security Tag [sn_sec_cmn_security_tag] table, in related list 'Applied Security Tags' you can see the M2M record
 

D) More complex logic

Here we need a bit of ingenuity.

Let's say we want to whitelist observable which has the same name as any non-retired windows server (for simplicity we will do it access-wise as simple as possible)

 

  1. Create a script include (we will use 'CustomObservableWhitelistingUtils') with 'Accessible from' set to 'All application scopes' and 'Sandbox enabled' = true
  2. Insert to it a function matchesWindowsServer which you can find after these steps
    1754925116657.png
  3. Go to the 'Enrichment: Whitelist' record on Security Tag [sn_sec_cmn_security_tag] table
  4. In Related list 'Security Tag Rules' click on 'New'
  5. Fill in the 'Name' (ie 'Matches Windows Server name'), keep in mind your company naming convention
  6. For table select 'Observable' [sn_ti_observable]
  7. In Condtion field select 'Value' and operator 'IS', then you need to call the Script Include function like this 'javascript&colon; new sn_sec_cmn.CustomObservableWhitelistingUtils().matchesWindowsServer(current.value);' Please replace script include name and scope with your own.
    1754924600222.png
  8. For testing Go to Windows Servers [cmdb_ci_win_server] table and select any one with Operational Status IS NOT retired - copy it's name
  9. Then go to the Observable table [sn_ti_observable] list
  10. Click on 'New'
  11. Paste the name of the Windows Server from step 8
  12. Submit the record
  13. Open the record you have just created
  14. See that it has tag 'Enrichment: Whitelist' attached to it. Also, if you open the XML of the record, you can see the sys_id of the tag in <security_tags>.
  15. Notice that also if you go to the 'Enrichment: Whitelist' record on Security Tag [sn_sec_cmn_security_tag] table, in related list 'Applied Security Tags' you can see the M2M record
  16. Also repeat the steps 8-13 with a retired windows server to confirm that the 'Enrichment: Whitelist' tag was not attached

 

/** 
 * Checks if the observableValue is same as name of any Windows Server in CMDB which is not retired 
 * @observableValue {String} Value of observable record 
 * @return {String} If found, returns the observableValue, else returns false 
 */
matchesWindowsServer: function(observableValue) {
    if (!gs.nil(observableValue)) {
        var serverGR = new GlideRecord('cmdb_ci_win_server');
        serverGR.addEncodedQuery('operational_status!=6^name=' + observableValue);
        serverGR.query();
        if (serverGR.hasNext()) return observableValue;
    }
    return false;
},
 

For custom matching of regex, we need to write our custom function and call it the same way. We will pass two arguments when calling a function from the condition field (observableValue and regex) and return the observableValue if it matched the regex (was satisfied) or false if not.

Code in the Script Include might then look like this:

 /** 
  * Checks whether value matches the regex 
  * @param {String} observableValue - value to test agains regex 
  * @param {object Regex} regex - Regular expression (ie. /^\d{1}$/g) 
  * @return {Boolean} - Returns observableValueif regex matches the value, else returns false 
  */
 checkValueAgainstRegex: function(observableValue, regex) {
     if (!gs.nil(observableValue) && !gs.nil(regex)) {
         try {
             regex = new RegExp(regex);
             if (regex.test(observableValue)) return observableValue;
         } catch (e) {
             return false;
         }
     }
     return false;
 },

Exclusion from integration capabilities

Now we have our Observables properly tagged but we still need to exclude them from the Integration Capabilities we don't want to get triggered.

Let's go for Observable Enrichment for showcase. We probably know the observable, since we have whitelisted it and therefore we don't need to collect pieces of information from external sources.

 

  1. Go to the Capabilities [sn_sec_cmn_capability] (Application Navigator -> Security Operations -> Integrations -> Integration Capabilities (Flows)
  2. Open 'Enrich Observable' record
  3. Notice the Related List 'Filter Conditions' and click 'New'
  4. Fill in 'Name' (ie. 'Filter whitelisted observables'), 'Condition Table' to 'Observable [sn_ti_observable]', then 'Conditions' - 'Security tags' CONTAINS 'Enrichment: Whitelist'
    1752188816505.png
  5. Submit the record
  6. Go back to 'Enrich Observable' Capability (step 1-2)
  7. If you have installed MISP, then in related list 'Capability Implementation' you should see a record for MISP - open it
  8. Set it to 'Active' = true (ticked) and Save (you might need to change the scope to 'MISP integration for Security Operations')
  9. For testing go to the Security Incident table [sn_si_incident] list
  10. Click on 'New'
  11. Fill in any 'Short Description'
  12. Submit the record
  13. Open the record you have just created
  14. In Related Links click on 'Show All Related Lists'
  15. Open Related List 'Associated Observables'
  16. Click 'Edit' button in the related list
  17. Filter records for 'Security tags' CONTAINS 'Enrichment: Whitelist'
  18. Select some IP Address and Domain Name (ie. 172.16.20.30 and google.com) and click on 'Save'
  19. Back on Security Incident record stay on 'Incident Details' tab and wait for a few minutes
  20. If you have MISP (or any other Observable Enrichment integration installed) you should be able to see result like on the image
    1752188690356.png

Exclusion from custom integrations

The story is a bit more complex here.

I would suggest add a condition at the beginning of flow or scripted logic which checks whether it has the appropriate tag ('Enrichment: Whitelist' in our case) and if so I would add a comment / worknote to the task (if it was run from the task) that it was Filtered based on the tag and end the flow / scripted logic.

Final touches

Finding = Clean

You might also want to mark whitelisted observable with Finding = 'Clean'.

For this you can do following:

 

  1. Go to Business Rule [sys_script] table
  2. Create a New Business Rule on Applied Security Tag [sn_sec_cmn_applied_security_tag]
  3. Set following values: Name = Any (ie. Set Finding for whitelisted observable), Description = Any, When = after (async can't access current object for delete), Insert = true, Delete = true, Condition = security_tag=e6e6ea180b503200263a089b37673a08^record_table=sn_ti_observable^EQ (Security Tag IS Enrichment: Whitelisted AND Record table IS sn_ti_observable), Script = You can find the script lower
  4. Submit the record
  5. For testing open any observable. Attach the 'Enrichment: Whitelisted' security tag to it
  6. Observe that Finding changed to 'Clean' (refresh might be needed)
  7. Remove the 'Enrichment: Whitelisted' security tag
  8. Observe that Finding changed to 'Unknown'(refresh might be needed)

 

(function executeRule(current, previous /*null when async*/) {
	var observableGR = new GlideRecord('sn_ti_observable');
	if (observableGR.get(current.getValue('record_id'))) {
		if (current.operation() == 'insert') 
			observableGR.setValue('finding', 'clean');
		else if (current.operation() == 'delete') 
			observableGR.setValue('finding', 'Unknown');		
		observableGR.update();
	}
})(current, previous);

Considerations

 

  • Be careful when using CONTAINS operator and regexes. This might create a blindspot which can be abused
  • Think about handing over the maintanance directly to Security Analyst teams to aleviate workload from developers and to streamline whitelisting observables and removing observables from whitelist

 

Specific tables used

 

  • Observables [sn_ti_observable]
  • Security Tag [sn_sec_cmn_security_tag]
  • Security Tag Rule [sn_sec_cmn_security_tag_rule]
  • Filter Group [sn_sec_cmn_filter_group]
  • Filter Condition [sn_sec_cmn_filter_condition]
  • Applied Security Tags [sn_sec_cmn_applied_security_tag]
  • Capabilities [sn_sec_cmn_capability]
0 Helpfuls
  • 115 Views
0 REPLIES 0