Security Operations email parsing

  • Release version: Yokohama
  • Updated January 30, 2025
  • 3 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Security Operations email parsing

    Security Operations email parsing enables ServiceNow customers to automatically generate Security Operations records—such as security incidents, vulnerabilities, and observables—from emails sent by external detection systems like malware detectors, firewalls, and threat intelligence tools. This integration streamlines threat response by ingesting security data directly into ServiceNow from email notifications.

    Show full answer Show less

    How Email Parsing Works

    • Each Security Operations plugin (Security Incident Response, Threat Intelligence, Vulnerability Response) has an emailto property defining the email address for receiving integration emails.
    • Incoming emails are stored in an email events table and processed against configured email parsers.
    • Emails that match a parser are flagged and transformed into or update existing Security Operations records, with the email linked to the record.
    • Unmatched emails are listed separately for review and parser improvement, with a reprocess option to retry parsing after updates.
    • Duplication rules prevent or manage multiple records for the same issue, defining whether to ignore, update, or create child records, and specifying which fields to update on duplicates.
    • Email parsing complements platform inbound actions but does not replace them and does not support indirect field updates like sysjournalfield entries.
    • By default, email events are deleted after 30 days to maintain system performance.

    Handling Multiple Records from One Email

    Emails can report on multiple items simultaneously. Using a configurable Record Separator within the email transform, the parser splits the email into sections, each evaluated separately to create multiple records as needed. For example, a malware report email can be split into sections listing different infected systems, generating a security incident record for each.

    • Common data applicable to all sections (e.g., Malware Hash or Malware Name) should be set to search anywhere in the email body.
    • Section-specific data (e.g., System name, IP address, Status) should be searched at the start of a line within each separated record section.
    • Records are created only for sections containing section-specific data; header or footer sections without such data do not create additional records but can contribute common field values to all records.

    Practical Use for ServiceNow Customers

    • Configure the emailto address to receive emails from external detection systems.
    • Create and customize email parsers and transforms to match your external email formats, enabling automatic creation or updating of Security Operations records.
    • Leverage duplication rules to maintain data quality and avoid redundant records.
    • Use the unmatched emails list and reprocess capabilities to iteratively improve parser accuracy.
    • Support for multiple records in a single email enables efficient handling of bulk alerts or reports, reducing manual effort.

    Next Steps

    • Create and customize email parsers within Security Operations to automate record creation from emails.
    • Edit and add transforms to refine data extraction from email content.
    • Review related concepts such as Security Operations enrichment data mapping and field mapping to enhance integration.

    Generate new Security Operations records from external detection systems using Email Parsing. This feature provides a method for integrating information from external tools such as malware detection, vulnerability detection, firewalls, threat intelligence, and more.

    How emails are parsed

    Any system that can send an email, can create Security Operations records, for example, security incidents, requests, vulnerable items, vulnerabilities, security incident observables, attack methods, and more.

    All Security Operations plugins (Security Incident Response, Threat Intelligence, and Vulnerability Response) have a property (email_to) that defines the email address where external integrations should send emails to, to be parsed by the email parsers. See Email Processing > Properties for more information.

    Email sent to any of the Security Operations email addresses is stored in an email events table. These emails are processed to determine whether they match any email parser.

    Emails that have a match are flagged and the transform and duplication rules create or update a Security Operations record. The email is linked to that record and flagged as matched.

    Emails that do not match are listed in Unmatched Emails as a Security Operations record. They can be reviewed to help build email parsers to handle these emails. A Reprocess action allows you to run the unmatched email through the parsers again. The original email log is linked to that record.

    The duplication rules for the email transform manage multiple emails relating to the same issue. These rules define what makes a duplicate record and can prevent duplicate records from being created. When a duplicate is detected, the rule specifies what action to take: no action (do not create a new record), create the new record as a child record of the existing record, or update the existing record. When updating the duplicate rule specifies which fields in the existing record are updated.
    Note:
    A Security Operations email parser works in conjunction with platform inbound actions and does not replace them. It does not support setting values on indirect fields for example, sys_journal_field entries.

    By default, email events are deleted after 30 days.

    Multiple records

    External detection systems (malware detectors, vulnerability, and so on) can send emails that report on multiple items at one time. The email parser supports separators within the email.

    For example, a malware detector could send you an email report about all systems within your network infected by one particular malware with information about the malware first, followed by a list of the systems affected.

    Figure 1. Sample malicious email
    Malicious email example
    In this example, when the Record Separator is set within your Email Transformas =================, it splits the email into four sections that are evaluated separately. This creates a Security Incident for each of the three affected systems.
    Note:
    The header section is detected but does not have any affected systems so, it is used in all three records and does not create a fourth record.

    Field Transforms pull in data from each section. If something in the header or footer of the email applies to all records, such as Malware Hash, Malware Name, and Type in this example, the field transform for them should set Search for value to a value that searches within the email body either At the start of a line in the email body or Anywhere in the email body.

    Field Transforms must be set to search At the start of a line within the record section or Sec for data that is defined within each section, such as System, IP address, or Status. The record section options are only available when there is a record separator defined within the email transform.

    When parsing an email with a separator defined, records are only created for sections with at least one piece of section-specific data.

    In this example, three records are created, even though there are four sections defined. The first section is a header, and it lacks anything specific to only one system. If any of the fields within the first section were filled in (System, IP, or Status), then a record would be created for that section, as well.