Workflow Data Fabric and GRC Indicators

[AlexR2]

We have started hearing about Workflow Data Fabric as a way to use external data for analysis in the platform without having to copy the data into the platform itself, our current approach. Has anyone experimented with WDF when it comes to GRC and control indicators? Would love to learn about your experience.

[Notorious BFG]

Hey @Simon Hendery !  While I can’t speak for the Risk BU, I can say that my limited experiments with using zero store queries both IRM indicators and PA has indeed worked.  That being said I’ve not attempted this in a production environment, only as part of a question I’d received in class that I was also curious to try out!  I can never resist an opportunity to rabbit hole in a live class setting so of course we went off road and realized it’s not difficult to run queries against these sources (at least the small demo sources I spun up anyway).  Again, not sure how it would play out in the “real world”, but my initial findings were positive!

If someone out there has a decently sized data bricks or other fabric source to play with I’d be happy to demonstrate the methods and specific use cases I used set up.  

[Abhi Sekhar · Accepted solution]

@AlexR2  This is a great question. I have recently completed an end-to-end experiment using Workflow Data Fabric (WDF) to integrate a Snowflake instance with ServiceNow GRC (IRM) Control Indicators.

The short answer is: It works seamlessly. You do not need to make any structural changes to the Control Indicator module; the integration relies entirely on how you configure the Data Fabric table.

Here is a detailed breakdown of the workflow and my findings from the experiment:

1. Connection and Schema Mapping

When you establish a new connection via Workflow Data Fabric, you gain direct visibility into the external source (in my case, Snowflake). You can see all schemas and tables that the connected service account has permission to access. This eliminates the need for traditional ETL or Import Sets.

2. Creating the Virtual Table

To use this data for GRC, you must create a Data Fabric Table within ServiceNow.

• Metadata Only: It is important to note that ServiceNow only saves the metadata (table structure and field definitions).

• Virtual Presence: The actual raw data remains in Snowflake. However, ServiceNow treats this as a "virtual" table, allowing it to appear in the dictionary and be selected within the GRC application.

3. Configuring the Control Indicator

Once the Data Fabric table is created, the process for a GRC administrator is identical to using a native table:

• Table Selection: In the Control Indicator record, you simply select your new Data Fabric table from the list.

• Logic Application: You can configure the relevant scripts or conditions exactly as you would for an internal ServiceNow table (e.g., Status IS Critical or Amount > 10000).

4. Execution and Audit Trail

The most critical part of this experiment was verifying the "Supporting Data" for audits.

• Live Execution: When the indicator runs, WDF queries the Snowflake data in real-time.

• Local Persistence: Although the source data stays in Snowflake, the Indicator Results capture the specific records identified during the run and save that supporting data locally in ServiceNow.

• Audit Readiness: This ensures that you have a point-in-time snapshot of the evidence required for audit purposes without having to maintain a massive local copy of the entire external database.

[AlexR2]

Thank you so much for your reply. Are there any limitations or constraints you could think of (i.e., volume of data, #of records, etc?). How's the latency on executing the indicator? We have use cases where we run indicators on tables containing 100k+ records.

As far as storing results in the local supporting data table, wouldn't it still have to "copy" the data to the instance and therefore storing the results impacting local storage?

[Notorious BFG]

@AlexR2 so this is where the “sample size” and “pass/fail” options come into play when setting up your indicators.  

If you’ve set up an indicator with a matching “fail” condition (meaning, if records in your source match the condition you’ve specified), and a sample size of “50”, up to 50 supporting data records matching this fail condition will be stored in the results of the indicator.  If you set up a “pass” condition with a sample size of 50, again, up to 50 records will be stored in the results.  

This is true even without using WDF, which is why we make sure in implementation to right size collection, set up appropriate archive and destroy rules (matching relevant data retention policies), and know when to use sample size “0” appropriately, which (counterintuitively) when selected actually stores EVERY matching record and checks against EVERY record in the target table.  

Many of us have been lobbying for a while to get the “Sample Size” field renamed to something like “matched record data collection threshold” or similar and to decouple it from the check all records vs the standard formula for 95% +/- 5% sample size given table record population as described in the indicatorUtils script include.  

[AlexR2]

We do have some control indicators where we select a sample but in other cases we want to test an entire population to ensure continuous compliance for a particular control (e.g. encryption at rest setting is turned on for every AWS SNS Topic). We have 2 indicators setup for a control, a pass side (requiring 100% pass) and a fail side (requiring no more than 0 failures). The pass side indicator shows our auditor the population that was tested so that they can ensure completeness of the population and accuracy, the fail side indicator shows if there were any failures identified. We can't select a sample in this particular control because there is no guarantee that a sample is representative of the population, encryption at rest is turned on on each AWS SNS Topic individually.