Security Operations Observable type mappings and creation

[ScottW1]

Hello community, 

We are wanting to cleanup and enrich our Observable data in order to effectively report details within the associated SIRs. I have the following goals. 

1. Focusing on 'unknown' observables (~4500 count) update the logic that maps the Observable Type to meet criterial specific to our environment. Currently the logic is not matching the value to an observable type and additional logic is needed to handle the observable. Not asking for the regex but how can I create that value to type transform? 
   1. An example might look like the value of "123abc.tif" and creating a regex that identifies this to Observable Type file. 
   2. Another example might look like the value of 'abcdefg' and creating a regex that identified this to Observable Type username. 
2. How can I create a custom Observable Type, with the intention on configuring a regex or script to map the value to that observable type. Expands on item 1 above but using a custom observable type. 

[AJ_UK]

Hi @ScottW1

I can't say I've done exactly what you are aiming to do yet, but...

I have seen the Sentinel Integration use Regex to validate observables against types before inserting them in the observable table sn_ti_observable, so defining a Regex pattern looks to be the way to go.

I suggest have a look at sn_ti.ObservableTypeIdentifier; sn_ti.ThreatUtils and it's _getObsTypeId method

and sn_sec_cmn.RegexValidationUtil which looks like where all the type matching happens.

Table sn_ti_observable_type appears to govern the allowable Observable types - this looks to be where you could create your custom observable type. Then add it's sys_id as an OBS_TYPE_ID definition in sn_ti.ThreatUtils

If you find out how to get it working, I'd be keen to know the methodology as I see a similar use case.