---
sourceDocument: Australia Workflow Data Fabric
sourceDocumentLink: https://www.servicenow.com/docs/r/australia/integrate-applications

 Release :

    - australia

ft:locale :

    - en-US

ft:publication_title :

    - Australia Workflow Data Fabric

ft:clusterId :

    - crint

bundleId :

    - crint

workflow :

    - Creator


---

# Viewing Kafka subscriptions and statistics

# Viewing Kafka subscriptions and statistics {#ariaid-title1}

Release version: Australia  
Updated March 12, 2026  
![](https://www.servicenow.com/docs/portal-asset/ico-clock) 5 minutes to read
Summarize  
![AI sparkle icon](https://servicenow.com/docs/portal-asset/ai-sparkle-icon) Summarized using AI  
This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.  

## Summary of Viewing Kafka subscriptions and statistics

This topic explains how to view detailed information about Kafka consumers and their performance in ServiceNow, specifically within IntegrationHub's Stream Connect.
It covers subscription records created when Kafka streams are activated, showing configuration, metrics, partition groups, logs, and consumer statistics.
This helps ServiceNow customers monitor and manage Kafka message consumption effectively.
Show full answer Show less  

## Viewing Subscription Records

To access a Kafka subscription record, navigate to **IntegrationHub \> Stream Connect \> Subscriptions**, select the info icon for the desired subscription, and open the record. Each subscription page includes:

* **Kafka subscription:** Displays consumer configuration like state (ACTIVE, REFRESHING, STOPPING), topic alias, consumer ID, auto position policy (earliest/latest), serialization format, error status, concurrency, partition count, and batch size.
* **Metrics:** Shows topic input rate, queue depth, consumer processing rate over three intervals (last 5 minutes, hour, 24 hours), plus estimated time to process the current queue.
* **Kafka subscription partition groups:** Details ownership status (NONE, PROCESSING, LOCKED), partition ranges, rates, and queue depth per partition group.
* **Stream Connect logs:** Provides log entries related to the consumer, including timestamps, message details, and links to related alerts or subscriptions. Detailed logging can be enabled temporarily via system properties.
* **Kafka consumer statistics:** Displays interval-based statistics such as offsets, lag, consumed/produced messages and bytes, linked to the consumer and partition group.

## Key Configuration and Metrics Explained

* **Subscription Configuration:** Includes crucial parameters like max concurrency, batch size, and serialization method that influence consumer performance.
* **Ownership States of Partition Groups:** Indicates which consumer owns and processes partitions, helping to troubleshoot processing delays or conflicts.
* **Stream Connect Logs:** Enable detailed logs temporarily with system properties to diagnose issues without overwhelming log storage.
* **Consumer Statistics Interval:** Adjustable logging interval and options to control logging behavior when no Kafka messages are processed, enabling tailored monitoring.

## Benefits for ServiceNow Customers

By using these views and metrics, customers can:

* Monitor Kafka consumer health and performance in real time.
* Identify processing bottlenecks through queue depth and lag analysis.
* Understand partition ownership for load balancing and troubleshooting.
* Access detailed logs for deeper diagnostics when needed.
* Customize monitoring intervals and logging behavior to fit operational needs.

This comprehensive visibility into Kafka subscriptions and consumer statistics empowers ServiceNow customers to optimize their Kafka integrations and ensure reliable message processing.  
View detailed information about a Kafka consumer and its performance. Details include the number of records added to the topic, the number of records processed, and the number of records remaining to be processed.

A subscription is a record associated with a consumer. It stores configuration and performance
details about the consumer. A subscription is created when a Kafka stream is activated.  
Each subscription page includes the following sections.

* Kafka subscription
* Metrics
* Kafka subscription partition groups
* Stream Connect logs
* Kafka consumer statistics
{#kafka-subscriptions-statistics__ul_u4b_2yg_lvb}

To view a subscription record, navigate to IntegrationHubStream ConnectSubscriptions. Select the info icon (![info icon]()) for the subscription you want to view, then select Open Record.  
Figure 1. Sample subscription page for a Kafka consumer

## Kafka subscription {#kafka-subscriptions-statistics__section_okk_3vg_lvb}

The Kafka subscription section shows configuration details for the consumer and subscription.{#kafka-subscriptions-statistics__table_scz_xzg_lvb__entry__2}

| Field | Description |
|-|-|
| Number | Number of the subscription record. |
| State | State of the consumer. Options are: * ACTIVE * REFRESHING * STOPPING {#kafka-subscriptions-statistics__ul_pc3_zsm_nvb} |
| Topic alias | Topic alias for the topic the consumer is processing messages from. |
| Consumer ID | Unique identifier for the consumer. |
| Kafka Auto Position Policy | Specifies where the consumer starts reading messages from the topic. Options are: * earliest: The consumer starts reading from the earliest available message in the topic. * latest: The consumer reads only the messages delivered to the topic after the consumer started. {#kafka-subscriptions-statistics__ul_xg4_zqm_nvb} |
| Serialization format | Serialization format for the message. Options are: * Plain Text * Encoded {#kafka-subscriptions-statistics__ul_h2t_zbr_1fc} |
| Has Error | Indicates if the subscription has any errors. |
| Max Concurrency | Maximum number of parallel processors used to consume messages and the number of partition groups. |
| Partition group timeout | Maximum time, in milliseconds, allocated to each partition group in the subscription. |
| Kafka partitions | Number of partitions in the topic. |
| Max Batch Size | Maximum number of messages processed per run. Options are: * automatically optimized: The system chooses the number of messages. * a number: The user manually selected a number. {#kafka-subscriptions-statistics__ul_dcy_vsm_nvb} |
[ ]

{#kafka-subscriptions-statistics__table_scz_xzg_lvb}

## Metrics {#kafka-subscriptions-statistics__section_wdb_lvg_lvb}

Each metric is measured over three time intervals: the previous five minutes, the previous hour, and the previous 24 hours.

* Topic Input Rate: The average number of records added to the topic per second.
* Topic Queue Depth: The average number of records remaining to be processed in the topic.
* Consumer Processing Rate: The average number of records processed per second. The Estimated time to process the current queue shows the time required to process the messages currently in the queue. It's calculated based on the queue depth and the consumer processing rate. The time estimate is formatted as HH:MM:SS.
{#kafka-subscriptions-statistics__ul_rkn_wvg_lvb}

## Related Links {#kafka-subscriptions-statistics__section_gpm_wcr_1fc}

View the record for the consumer that is processing the messages by selecting Open consumer processing record.

## Kafka subscription partition groups {#kafka-subscriptions-statistics__section_crp_mvg_lvb}

The Kafka subscription partition groups tab shows the following details.{#kafka-subscriptions-statistics__table_m42_ckn_nvb__entry__2}

| Field | Description |
|-|-|
| Number | Number for the partition group. |
| Owner of this partition group | Ownership state of the partition group. Options are: * NONE: The partition group isn't owned by any Kafka consumer jobs. It's available to pick up and process. * PROCESSING: The partition group is owned by a Kafka consumer job and is processing messages. * LOCKED: The partition group isn't owned by a Kafka consumer job and isn't available to pick up. {#kafka-subscriptions-statistics__ul_kl4_xxn_nvb} When a Kafka consumer job is activated, it checks for the earliest updated record with an owner of NONE and picks one. Then the consumer job changes the owner to PROCESSING and processes it for 10 seconds. When it's done processing, the state is changed back to NONE, so it can be picked up and processed again later. If the consumer job processes all the messages, it sets the owner to LOCKED for 10 seconds, after which it's set to NONE again. |
| Updated | Date and time the partition group was last updated. |
| When is the ownership lost | Date and time that ownership is lost. |
| First partition | Number of the first partition for the partition group. |
| Last partition (inclusive) | Number of the last partition for the partition group. |
| Input rate Given for the last five minutes, last hour, and last 24 hours. | Average number of records added to the partition group per second. |
| Consumer processing Given for the last five minutes, last hour, and last 24 hours. | Average number of messages consumed from the partition group per second. |
| Queue depth Given for the last five minutes, last hour, and last 24 hours. | Average number of records remaining to be processed in the partition group. |
[ ]

{#kafka-subscriptions-statistics__table_m42_ckn_nvb}

## Stream Connect logs {#kafka-subscriptions-statistics__section_pqk_qvg_lvb}

The Stream Connect Logs tab displays the log entries for the associated consumer. Each log entry shows when it was created; its level, message, and source; and links to any related Alert or Subscription records. Stream Connect logs are stored in the Stream Connect Logs \[sys_consumer_log\] table. This table also includes logging for [producers.](https://www.servicenow.com/docs/nz1XsKjMy7tvHV2hSAqqqg "View detailed information about a Stream Connect producer and its performance, including the producer type and ID, and the total number of bytes and messages produced to a topic.")

You can enable more detailed logging in the Stream Connect logs by [adding the system property](https://www.servicenow.com/docs/access?context=t_AddAPropertyUsingSysPropsList&version=australia&pubname=australia-platform-administration&ft:locale=en-US)
glide.ih.kafka.stream_connect.debug and setting it to true. To avoid filling up the logs, this property is automatically disabled after 24 hours. If you need detailed logging for
longer than 24 hours, you can re-enable the property manually.

Use the glide.ih.kafka.producer.message_bytes_to_log property to specify how many bytes of a message to display in the logs. This property only impacts logging for producers. The default value is 0, meaning
no message content is logged.

## Kafka consumer statistics {#kafka-subscriptions-statistics__section_ggw_qvg_lvb}

The Kafka Consumer Statistics tab shows the following details.{#kafka-subscriptions-statistics__table_s2d_ygn_nvb__entry__2}

| Field | Description |
|-|-|
| Created | Date and time the consumer statistics record was created. |
| Partition group | Record for the partition group. |
| Topic Alias | The topic alias for the topic associated with the consumer. A topic alias is a unique topic name that can be connected to any underlying Hermes or Direct Kafka topic. |
| Topic | The Hermes or Direct Kafka topic associated with the consumer. |
| Interval | Duration for the consumer statistics record, given in seconds. You can change the length of the interval with the glide.ih.kafka.consumer.log_interval property. The default value is 60 seconds. You can specify whether to log messages to the consumer log when there are no produced or consumed Kafka messages during the interval with the glide.ih.kafka.consumer.log_empty_stats property. The default value is true. |
| Offset | Sum of all the offsets for the partition group. |
| End offset | Sum of all the end offsets for the partition group. |
| Lag messages | Difference between the end offset and the current offset. |
| Consumed messages | Number of messages processed by the consumer during the interval time. |
| Produced messages | Number of messages produced to the topic during the interval time. |
| Consumed bytes | Number of bytes processed by the consumer during the interval time. |
| Consumer ID | Unique identifier for the consumer. |
| Consumer Table | Name of the underlying consumer table. One of the following: * Kafka Stream \[sys_kafka_stream\] * Kafka Flow Trigger \[sys_kafka_flow_trigger\] {#kafka-subscriptions-statistics__ul_hmh_xsr_4hc} |
[ ]

{#kafka-subscriptions-statistics__table_s2d_ygn_nvb}

*[\>]: and then