Log correlators for identifying related alerts
Summarize
Summary of Log correlators for identifying related alerts
In Health Log Analytics, log correlators are critical for detecting relationships between alerts by analyzing keys or values in log data. They help determine if multiple alerts are part of a larger, related issue, enhancing your ability to troubleshoot and manage incidents effectively.
Show less
Key Features
- Types of Log Correlators:
- Free Text Correlators: Analyze the free-form text in log messages to detect correlation terms not extracted as structured properties, such as unique system, application, or service names. For example, adding a service name like "teatime" as a free text correlator helps identify alerts across components related to that service.
- Log Property Correlators: Analyze structured metadata in log lines, such as service instance names, interface IDs, or request IDs. These correlators identify related alerts based on specific business context and metadata values appearing across different log sources.
- Scope of Application: You can configure correlators to analyze:
- Only new log sources created after activation of the correlator
- All existing log sources
- A specific log source of your choice
- Customization: The base system provides several default correlators, but you can define custom correlators tailored to your environment’s needs.
- Exclusion Capability: You can exclude certain log sources from analysis by a correlator to refine alert correlation and avoid noise from irrelevant sources.
Key Outcomes
By implementing log correlators in Health Log Analytics, you can efficiently identify related alerts across multiple log sources, streamline incident detection, and gain deeper insights into the root causes affecting your services. This capability improves operational awareness and accelerates troubleshooting in complex environments.
In Health Log Analytics, log correlators are keys or values in log data that detect correlations between alerts. These correlations help you determine whether an alert is part of a larger issue.
For example, a log correlator could detect when the interface ID of a particular network device occurs simultaneously in multiple warnings across service instances.
You can identify related alerts in your log data by using log correlators. The base system includes several log correlators. You can define custom correlators for a specific log source, all log sources, or only log sources created after the correlator is activated.
Most log lines include a metadata portion plus a message portion. Some log lines, however, include only message text with metadata included in the text. The two types of log correlators, free text correlators and log property correlators, analyze different portions of each log to identify relationships between data from multiple sources.
- Free text correlators
-
Free text correlators analyze the text within the log message portion of log lines that are associated with an anomaly. The system uses free text correlators to identify correlations between alerts. You use free text correlators to add a term that you expect to appear within log messages. A good choice is a term that is not structured and would not otherwise be extracted as a log property. For example, “policy-id” or “ thread-id”.
You also typically add free text correlators for the names of systems, applications, and services that are unique to your environment. Because such a value can be referred to by multiple sources, layers, middleware, or databases, the free text correlator can be an effective detector of correlated alerts. For example, if your organization's service is called TeaTime, then you might add "teatime" as a free text correlator. The correlator would identify alerts that are related because they were generated for resources that support the TeaTime service. For example, a database lock or a connection failure between TeaTime components.
- Log property correlators
-
Log property correlators analyze the metadata portion of log lines. For example, the correlator can analyze the name of a service instance, the interface ID of a network device, or the request ID of a web-facing component. A log property correlator could flag a correlation when the interface ID of a network device simultaneously occurs in multiple warnings in different log sources. Log property correlators are specific to the business context of your environment.
- Only new sources: The system applies the log correlator only to log lines from log sources that were created after this log correlator is activated.
- All sources: The system applies the log correlator to log lines from all log sources.
- Specified source: For a log correlator, the system analyzes only log lines from the log source that you specify.