What is log analytics?

A log analysis can unveil patterns or anomalies in user behaviours, identify issues, and reveal security issues.

Analysing logs provides the opportunity for active monitoring to review application performance, behaviours, and any anomalies across the application. Proactive monitoring can help a team identify any issues that didn’t trigger an alert or have a unique explanation. Regardless of whether or not it triggers an alert, it will show up in the log data.

Aggregated and structured data can provide the opportunity for troubleshooting at all levels. Log analytics provide a baseline for standard activity, which can help determine why something may have deviated from the baseline.

IT teams have visibility into logs and metrics using a high-level dashboard that provides a unified view of the information for easier analysis. The dashboards then provide the ability to highlight KPIs, SLAs, and other necessary statistics using data from a log analysis.

Log analysis provides the benefit of insights into trending data and growth rates. A histogram can help visualise a rate of growth that can assist with lifecycle management and capacity planning.

It is important to use data that is usable and accurate. There is the potential for data to become corrupted if:

• The storage disk crashes.
• Applications aren’t terminated properly.
• A virus infects the system.
• There are input/output configuration issues.

Data tends to use different naming conventions, as it is collected from a variety of sources. It is important to correlate the data from the various sources and standardise the terminology to reduce confusion and error during analysis.

The data is ready to be reviewed once it has been collected, cleaned, and organised. There are various methods of analysis depending on the processes, intended use of the data, and the size of the data sets. Some of the options include:

• Classification: It can be easier to filter and adjust data when it is labelled with keyword tags that organise the data into different categories.
• Pattern recognition: Detecting patterns by filtering messages can assist with the recognition of data patterns that may assist in the detection of anomalies.
• Artificial ignorance: Data can be more difficult to sift through with routine log messages increasing the density. Artificial ignorance is a machine learning system that ignores more routine updates unless they did not occur.
• Correlation analysis: It can be ineffective to gather information from servers, operating systems, and network devices when there isn’t a way to compare the data in the event that there is a single system-wide event. Correlation analysis works through messages from all components related to an event.

This is also referred to as multi-dimensional anomaly detection.

Organisations can respond to security threats—including intruder and DOS threats—more quickly and find the root cause more efficiently. Future events can be prevented once a root cause is identified and the issue is reconciled.

Multiple departments rely on IT to carry out their tasks and responsibilities. Log analysis provides the ability to spot system errors or issues before they result in an outage, and address them efficiently and quickly. Log analysis is also a part of maintaining Service Level Agreements between IT teams, other departments, and customers. Proactivity helps avoid service interruptions and downtime of products, which can lead to lost revenue.

Organisations and individual teams can better their decision making process, evaluate their strategies, and make any adjustments as necessary. Log analysis makes all of this possible.

Logs provide the opportunity to gather information on conversions, traffic volume, and how visitors navigate around a site. This provides the opportunity to analyse customer interactions and determine whether or not the customer experience could be improved or if the sales team needs to adjust messaging.

It’s important to access as much information as possible when something fails. There are two types of monitoring applications:

• Rules-based: When the things that should be monitored are identified, log analysis tools can be used to identify errors for application optimisation. A team can create their own set of rules to generate alerts using a series of different channels. But rules-based relies on people knowing all the possible issues that could occur in advance, which does not scale in today’s digital services environment.
• Machine learning log analysis: Machine learning tools can automatically detect problems and identify anomalies across the IT eco-system, including an application behaviour. The tool scans through the data and interprets issues that could possibly occur.

The ServiceNow Health Log Analytics feature, part of ITOM Predictive AIOps, alerts to early signals of potential outages.

With ServiceNow ITOM Health Log Analytics, you can:

• Get information on anomalies without having to establish any pre-defined thresholds.
• Get visibility into issues that result from Blind Spots that you could not have anticipated in advance.
• Get root-cause information in near real time, which significantly reduces the Mean Time To Repair (MTTR).
• Take action on alerts.
• Add alert rules.
• Mark alerts as significant.
• Filter to reduce noise.
• View and adjust the status of system health notifications.
• Add a log correlator to identify relationships in logs.