Monitoring log data flow and optimizing integration settings in Health Log Analytics

  • Release version: Australia
  • Updated March 12, 2026
  • 4 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Monitoring Log Data Flow and Optimizing Integration Settings in Health Log Analytics

    The Overview screen in Health Log Analytics provides insights into the log-processing pipeline of active integrations, enabling customers to monitor streaming status and troubleshoot issues. It gives direct access to essential features like Data Input Mapping, Source Type Structures, and Log Sources pages, along with the Log Viewer contextually linked to the current integration.

    Show full answer Show less

    Key Features

    • Streaming Status: Displays the real-time streaming status of logs between the log source, MID Server, and HLA engine, including logs per minute metrics. Note that the ServiceNow System Logs Retriever integration does not utilize a MID Server.
    • Alert Metrics: The AIOps component indicates total alerts created by the HLA engine, with live data updated every minute.
    • Error Handling: If streaming fails, the integration deactivates, highlighting the failure point and offering troubleshooting steps or support options.
    • Log Streaming Sources: Lists active log sources streaming data, including status, last event time, and log processing metrics.

    Key Outcomes

    By effectively using the Overview screen, ServiceNow customers can ensure continuous data flow into their instances, allowing for timely anomaly detection and alert generation. For instance, if an Elasticsearch integration shows a failure, resolving credential issues and restoring connections quickly enables HLA to resume its critical functions. This proactive monitoring and adjustment capability is essential for maintaining the integrity and responsiveness of log data management.

    The Overview screen in Health Log Analytics provides a comprehensive view of the components in the log-processing pipeline of a specific active integration. From this screen, you can troubleshoot any streaming issues for this integration and adjust its settings if needed.

    The Overview screen shows the log data streaming status and streaming sources of an active integration. It provides direct access to the Data Input Mapping, Source Type Structures, and Log Sources pages, as well as the Log Viewer, all with context from the current integration.

    Figure 1. Overview screen and View menu items
    Integration Overview screen and View menu items.

    Streaming status

    The Streaming status section on the Overview screen shows the status of data streaming between the log source and the MID Server and between the MID Server and the HLA engine. In addition, it displays the number of logs per minute passing through the MID Server and through the HLA engine.
    Note:
    For the ServiceNow System Logs Retriever integration, the Overview screen doesn’t display the MID Server streaming status because that integration doesn't run on a MID Server.

    The ServiceNow AIOps component shows the total number of alerts that the HLA engine has created. These statistics are updated when the Overview screen loads and are automatically refreshed every minute to show live data. You can change the default auto-refresh time interval through the system property sn_itom_integ_app.overview_page_data_input_stats_auto_refresh_interval_seconds.

    If data streaming fails, the integration is automatically deactivated and the Streaming status marks the component where the failure occurred. In addition, a banner explains the failure and either proposes steps to take to fix it or refers to ServiceNow support.

    Streaming status failure.

    For MID-less or OpenTelemetry Protocol (OTLP) integrations, such as Amazon Data Firehose, the Overview screen displays the ITOM Gateway as a component in the log-processing pipeline. The MID Server component is not shown in the pipeline, because log data is sent directly from the source to the ITOM Gateway. The logs are then processed by the HLA engine to find anomalies. Overview screen displaying the ITOM Gateway component.

    For these integrations, the Overview screen shows the average rate of logs per minute over the last 15 minutes passing through the ITOM Gateway and the HLA engine, similar to the metrics shown for MID-based ingestion.
    Note:
    The ITOM Gateway component is shown only if the MID Server property mid.hla.itom_gateway_streaming.enabled is set to true. For more information, see MID Server properties.
    When the integration isn't streaming live data from the source, a warning message is displayed. There is a 4-hour interval between the last log source time and the current time before this message is generated. You can change the default time interval through the system property sn_itom_integ_app.overview_page_log_source_time_threshold_hours.
    Note:
    The warning message is displayed only when the HLA engine is up and running.

    Log streaming sources

    The Log streaming sources section displays all sources that are currently streaming log data to HLA through this integration. Depending on your integration, the Log streaming sources table displays the following information about the integration's log streaming sources.
    Note:
    This table isn’t available for the ServiceNow System Logs Retriever integration.
    Table 1. Log streaming sources
    Column Description
    Name The name of the log data source.
    Status The streaming status: Active or Not active.
    Data input The integration streaming the data to your ServiceNow instance.
    MID Server The MID Server to which the log data is streaming.
    Last event time The time when the integration received the latest event.
    Last log processing time The time when the last log was received or processed.
    Raw log lines/sec The average number of raw log lines that streamed to the MID Server per second in the last one-minute interval.
    Note:
    This value represents the number of raw log lines before preprocessing.
    Pre-processed log lines/sec The average number of preprocessed log lines that streamed to the MID Server per second in the last one-minute interval.
    Note:
    This value can differ from the number of raw log lines per second. For example, the difference can be a result of logs having been dropped during preprocessing.

    Example

    As an admin, you use the integration Overview page to verify continuous data flow into the ServiceNow instance. If log streaming fails, HLA doesn't receive the real-time data needed to detect anomalies or generate alerts.

    For example, the Streaming Status for an active Elasticsearch integration may show a red circle with a white X for the MID Server component in the log-processing pipeline. This indicates that logs are not reaching the MID Server.

    The Log Streaming Sources table shows a state of Authentication Failed or Connection Error, with an error message indicating invalid credentials. This points to an issue with the Elasticsearch credentials. Fixing the credentials and either restarting the log service or waiting for the next polling cycle, restores the connection.

    The Log Streaming Sources table now shows a connection state of Connected or Active with successful authentication. The Streaming Status displays a green circle with a white check mark for the MID Server component. With log streaming restored, HLA can resume processing data and generating anomaly alerts.