Identifying relationships in log data by using log correlators

  • Release version: Yokohama
  • Updated January 30, 2025
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Identifying relationships in log data by using log correlators

    Log correlators in ServiceNow Yokohama release help identify relationships between alerts by detecting correlations within log data. They enable you to determine if an alert is part of a broader issue by analyzing keys or values across multiple log sources. This capability enhances troubleshooting by grouping related alerts that share common attributes, such as network device interface IDs or service instance names.

    Show full answer Show less

    Key Features

    • Types of Log Correlators:
      • Free Text Correlators: Analyze unstructured terms within the log message portion of log lines, useful for identifying correlations based on unique names or identifiers (e.g., “policy-id”, service names like "TeaTime"). These are effective when values span multiple systems, middleware, or databases.
      • Log Property Correlators: Analyze structured metadata within log lines, such as service instance names, interface IDs, or request IDs. These correlators are tailored to your business context and can detect simultaneous occurrences of key metadata across different log sources.
    • Scope Configuration: You can specify which log sources the correlators apply to:
      • Only new log sources: Applied only to log sources created after activation of the correlator.
      • All log sources: Applied to every log source.
      • Specified log source: Applied only to a chosen log source.
    • Custom and Base Correlators: The system includes several pre-defined correlators, and you can create custom correlators to fit specific needs or environments.
    • Excluding Sources: You can exclude specific log sources from analysis by a correlator to reduce noise or focus on relevant data.

    Key Outcomes

    • Improved detection of related alerts by correlating log data across multiple sources and layers.
    • Faster identification of root causes for complex issues involving multiple components or services.
    • Customizable correlation rules tailored to unique organizational environments and business contexts.
    • Enhanced operational efficiency by automatically grouping related alerts, reducing manual investigation efforts.

    Log correlators are keys or values in log data that detect correlations between alerts to help you determine whether an alert is part of a larger issue. For example, a log correlator could detect when the interface ID of a particular network device occurs simultaneously in multiple warnings across different service instances.

    You can identify related alerts in your log data by using log correlators. The base system includes several log correlators, and you can define custom correlators for a specific log source, all log sources, or only log sources created after the correlator is activated.

    Most log lines include a metadata portion plus a message portion. Some log lines, however, include only message text with metadata included in the text. The two types of log correlators, free text correlators and log property correlators, analyze the different portions of each log to identify relationships between log data from multiple log sources.

    Free text correlators

    Free text correlators analyze the text within the log message portion of log lines that are associated with an anomaly. The system uses free text correlators to identify correlations between alerts. You use free text correlators to add a term that you expect to appear within log messages. A good choice is a term that is not structured and would not otherwise be extracted as a log property. For example, “policy-id” or “ thread-id”.

    You also typically add free text correlators for the names of systems, applications, and services that are unique to your environment. Because such a value can be referred to by multiple sources, layers, middleware, or databases, the free text correlator can be an effective detector of correlated alerts. For example, if your organization's service is called TeaTime, then you might add "teatime" as a free text correlator. The correlator would identify alerts that are related because they were generated for resources that support the TeaTime service, such as a database lock or a connection failure between TeaTime components.

    Log property correlators

    Log property correlators analyze the metadata portion of log lines. For example, the correlator can analyze the name of a service instance, the interface ID of a network device, or the request ID of a web-facing component. A log property correlator could flag a correlation when the interface ID of a network device simultaneously occurs in multiple warnings in different log sources. Log property correlators are specific to the business context of your environment.

    You can specify the set of log sources whose log data are analyzed by a log correlator. Choices are as follows:
    • Only new sources: The system applies the log correlator only to log lines from log sources that were created after this log correlator is activated.
    • All sources: The system applies the log correlator to log lines from all log sources.
    • Specified source: For a log correlator, the system analyzes only log lines from the log source that you specify.