Security Operations email parsing
Summarize
Summary of Security Operations email parsing
Security Operations email parsing enables ServiceNow customers to automatically generate Security Operations records from emails sent by external detection systems such as malware detectors, vulnerability scanners, firewalls, and threat intelligence tools. This integration streamlines the ingestion of security-related data, allowing for quicker threat detection, response, and remediation within ServiceNow Security Operations.
Show less
How Email Parsing Works
- Emails sent to designated Security Operations email addresses (configured via the emailto property) are stored in an email events table.
- The system processes these emails by matching them against defined email parsers, which extract relevant data to create or update Security Operations records such as security incidents, vulnerabilities, and observables.
- Matched emails are linked to created or updated records and flagged accordingly.
- Emails that do not match any parser are listed as unmatched and can be reviewed to improve parser definitions; unmatched emails can be reprocessed once parsers are updated.
- Duplication rules control how multiple related emails are handled, preventing duplicate records by defining whether to ignore, update, or create child records, and specifying which fields to update.
- Email parsing complements but does not replace platform inbound actions and does not modify indirect fields like journal entries.
- By default, processed email events are deleted after 30 days to manage storage.
Handling Multiple Records in a Single Email
The email parser supports splitting an email into multiple record sections using a defined Record Separator. This allows emails reporting multiple affected systems or vulnerabilities in one message to be parsed into individual Security Operations records.
- Each section separated by the defined delimiter (e.g., "=================") is evaluated separately.
- Fields that apply universally (like Malware Name or Hash) can be extracted from header or footer sections and applied to all records.
- Section-specific data (e.g., affected system, IP address, status) is extracted only from the relevant sections.
- Records are created only for sections containing at least one piece of section-specific data; header-only sections without specific details do not generate records but provide contextual information.
Creating and Editing Email Parsers
ServiceNow customers can create and customize email parsers to tailor how emails are transformed into Security Operations records. This includes defining field transforms to extract specific data points and setting parsing rules to handle complex email formats.
Editing existing email transform rules allows refinement of data ingestion, improving accuracy, and response efficiency.
Benefits for ServiceNow Customers
- Automates the intake of security alerts and findings from diverse external tools into a centralized Security Operations environment.
- Reduces manual effort and accelerates incident and vulnerability management by directly generating actionable records from emails.
- Supports complex email formats reporting multiple issues simultaneously, ensuring comprehensive coverage.
- Enables ongoing improvement through unmatched email review and reprocessing capabilities.
Generate new Security Operations records from external detection systems using Email Parsing. This feature provides a method for integrating information from external tools such as malware detection, vulnerability detection, firewalls, threat intelligence, and more.
Any system that can send an email, can create Security Operations records, for example, security incidents, requests, vulnerable items, vulnerabilities, security incident observables, attack methods, and more.
All Security Operations plugins (Security Incident Response, Threat Intelligence, and Vulnerability Response) have a property (email_to) that defines the email address where external integrations should send emails to, to be parsed by the email parsers. See for more information.
Email sent to any of the Security Operations email addresses is stored in an email events table. These emails are processed to determine whether they match any email parser.
Emails that have a match are flagged and the transform and duplication rules create or
update a Security Operations record.
The email is linked to that record and flagged as matched.
Emails that do not match are listed in Unmatched Emails as a Security Operations record. They can be reviewed to help build email parsers to handle these emails. A Reprocess action allows you to run the unmatched email through the parsers again. The original email log is linked to that record.
By default, email events are deleted after 30 days.
External detection systems (malware detectors, vulnerability, and so on) can send emails that report on multiple items at one time. The email parser supports separators within the email.
For example, a malware detector could send you an email report about all systems within your network infected by one particular malware with information about the malware first, followed by a list of the systems affected.
Field Transforms pull in data from each section. If something in the header or footer of the email applies to all records, such as Malware Hash, Malware Name, and Type in this example, the field transform for them should set Search for value to a value that searches within the email body either At the start of a line in the email body or Anywhere in the email body.
Field Transforms must be set to search At the start of a line within the record section or Sec for data that is defined within each section, such as System, IP address, or Status. The record section options are only available when there is a record separator defined within the email transform.
When parsing an email with a separator defined, records are only created for sections with at least one piece of section-specific data.
In this example, three records are created, even though there are four sections defined. The first section is a header, and it lacks anything specific to only one system. If any of the fields within the first section were filled in (System, IP, or Status), then a record would be created for that section, as well.