Configure an Apache Kafka data input in Health Log Analytics manually

  • Release version: Australia
  • Updated March 12, 2026
  • 3 minutes to read

    • Set up a data input for streaming log data from Apache Kafka to your ServiceNow instance.

    Before you begin

    • Verify that a MID Server is installed and configured with the Log Ingestion capability enabled. For more information, see MID Server system requirements.
      Important:
      Health Log Analytics does not support IPv6. To work with the application, configure the MID Server to IPv4.
    • Unless the MID Server and external clients are on the same network, the MID Server must have a public IP address. This is required when its IP is exposed through network address translation (NAT), a load balancer, or a similar device. The public IP address enables external clients, such as Filebeat agents located outside its network, to reach the MID Server. Private IP addresses are not routable over the internet. Without a public IP, external clients cannot connect to the MID Server even if they are configured with its address. In the MID Server properties, add a property named mid.public_ip with the public IP address as the value. For more information, see Create a MID Server property. If the MID Server and external clients are on the same network, connections can be made using the private IP address.

    Role required: evt_mgmt_admin

    Procedure

    1. Navigate to All > Health Log Analytics > Data Input > Data Inputs.
    2. On the Data Inputs page, select New.
    3. Choose the Apache Kafka data input.
    4. On the form, fill in the fields.
      For a description of the fields, see Apache Kafka data input configuration fields.
    5. Optional: Select Advanced to set additional configuration fields.
      For a description of the fields, see Apache Kafka data input configuration fields. For information about configuring the advanced settings later, see Configure advanced settings for Apache Kafka data inputs in Health Log Analytics manually.
    6. On the Query Settings tab, fill in the field.
      For a description of the fields, see Apache Kafka data input configuration fields.
    7. On the Transport tab, fill in the fields.
      For a description of the fields, see Apache Kafka data input configuration fields.
    8. Select Save.
      Health Log Analytics adds the data input record to the Data Inputs table.
    9. Verify that the data input is configured correctly by selecting Test connection.

      Health Log Analytics tries to connect the MID Server to the data repository.

      If the data input is configured to run on a MID Server cluster, the system tries to connect all the MID Servers contained in the cluster to the repository. The cluster passes the test if at least one of its MID Servers gets connected. This feature is supported in the Health Log Analytics application, Version 26.0.17 - February 2023 and later, available from the ServiceNow Store.

      • If the connection was established, the Test connection button is turned off and the Publish button is enabled.
      • If the connection failed, the reason for the failure displays in the Error message field. Resolve the issue, select Save if you modified the configuration, and then select Test connection to test the connection again.
        Note:
        You can only publish the data input configuration when the connection is created successfully.
      Note:
      You can revert to the last published configuration by selecting Revert Changes. This option is available only when you're modifying a configuration that has been published previously.
    10. Select Publish to publish the data input to the MID Server.

    Result

    The data input configuration process is complete. Health Log Analytics adds the data input record to the Data Inputs table and attaches the configuration file to the data input record. The data input starts streaming log data from Apache Kafka to your ServiceNow instance.

    Note:
    If the HLA engine is down and data has stopped streaming, a notification appears at the top of the data input configuration page. When this happens, contact ServiceNow support.

    What to do next

    Make sure that the data input is streaming data.