Health Log Analytics configuration preferences

  • Release version: Washingtondc
  • Updated February 1, 2024
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Health Log Analytics Configuration Preferences

    This section outlines essential settings and configurations for Health Log Analytics within IT Operations Management. It emphasizes the importance of proper MID Server settings for optimal log ingestion performance.

    Show full answer Show less

    Key Features

    • MID Server Settings: Enable log ingestion capability on the MID Server. Use dedicated MID Servers for this purpose when possible.
    • Preferred Specifications: Recommended configurations include:
      • CPUs: 8
      • RAM: 32 GB
      • Network Bandwidth: Up to 10 Gbps
      • EBS Bandwidth: Up to 4,750 Mbps
      • Maximum Java heap size: 8,192 MB
    • Log Ingestion Throughput: Throughput expectations vary by log message size, with specific rates provided for 300 bytes, 1.1 KB, and 2 KB messages.
    • Minimum Requirements: For streaming logs, the minimum settings include:
      • CPUs: 4
      • RAM: 16 GB
      • Java heap size: 8 GB
    • Ulimit Configuration: Adjust ulimit settings for open files on the MID Server to enhance throughput, though the correlation with throughput cannot be modified.
    • Data Input Limits: Default input per MID Server is limited to 10, which can be configured as needed.
    • Java Runtime Requirement: MID Servers must operate on JRE 11 or above in both FIPS and non-FIPS modes.
    • Log Source Retention: Default retention is three days, with options to modify this setting for specific sources starting from Version 22.0.12.

    Key Outcomes

    By properly configuring the MID Server and understanding the log ingestion parameters, customers can significantly enhance log processing efficiency and ensure optimal performance of the Health Log Analytics application. This enables better monitoring and analysis of operational health data.

    Commonly used settings for Health Log Analytics properties and general configuration.

    MID Server settings

    • The MID Server log ingestion capability must be enabled.
      Note:
      Enabling All capabilities on the MID Server includes enabling the log ingestion capability.
    • Use dedicated MID Servers for log ingestion whenever possible.
    • To enable MID Servers to run multiple products, Health Log Analytics must have at least the Java Virtual Machine (JVM) memory setting for the standard product for each MID Server thread configuration.
    The preferred MID Server settings for Health Log Analytics are:
      • CPUs: 8
      • RAM: 32 GB
      • Network Bandwidth: Up to 10 Gbps
      • EBS Bandwidth: Up to 4,750 Mbps
      • Maximum Java heap size for MID Server: 8,192 MB
      With the above specifications, the expected log ingestion throughput on a Washington DC instance is as follows:
      • For a log message of 300 bytes: 20,000
      • For a log message of 1.1 KB: 12,300
      • For a log message of 2 KB: 7,970
      The minimum requirements for streaming logs to Health Log Analytics are:
      • CPUs: 4
      • RAM: 16 GB
      • Java heap size for MID Server: 8 GB

      For general information, see: MID Server system requirements.

    • To increase log ingestion throughput, you can either increase the ulimit or the network bandwidth, or decrease the size of the logs being streamed. The ulimit setting can be configured on an individual MID Server. However, the correlation between the ulimit and the throughput can’t be modified.

      The following table lists the ulimit settings for open files relating to network throughput on the MID Server. It shows the size of the logs being streamed from the MID Server to the agent, and the gRPC streaming rate equivalent to the throughput.

      Table 1. Ulimit settings in relation to throughput
      Queue Type Log line size gRPC rate
      In Memory Queue 300 bytes 18,000
      In Memory Queue 1.1 KB 13,000
      In Memory Queue 2 KB 10,000
      Disk-based Queue 300 bytes 11,000
      Disk-based Queue 1.1 KB 5,000
      Disk-based Queue 2 KB 3,000
    • By default, the number of data inputs per MID Server is limited to 10. You can configure this limitation for an individual MID Server or for all MID Servers.
    • Both in FIPS and non-FIPS mode, MID Servers with Health Log Analytics capability must run on the Java Runtime Environment (JRE) 11 or above.

    Log source retention settings

    By default, log retention per source is set to three days. This setting can't be modified.

    When using Health Log Analytics application, Version 22.0.12 - December 2021 and later, available from the ServiceNow Store , you can modify the log retention policy per source or for multiple sources together. For more information, see Modify the log source retention period.