Using the Stream Connect Dashboard

View detailed statistics for your Stream Connect integrations, and create or edit topics, replicators, and consumers with the ServiceNow® Stream Connect dashboard.

The Stream Connect Dashboard combines several capabilities into a single environment, so you can:

• Manage multiple Stream Connect integrations through a single application.
• Gain insights into how your integrations use and process data with detailed charts, lists, and statistics.
• Create and edit topics, replicators, and consumers through a simplified interface.

Request apps on the Store

Visit the ServiceNow Store website to view all the available apps and for information about submitting requests to the store. For cumulative release notes information for all released apps, see the ServiceNow Store version history release notes.

Automation Engine subscription

This application requires an Automation Engine subscription and a Stream Connect subscription. For more information, see https://www.servicenow.com/products/automation-engine.html.

Stream Connect Dashboard organization

To view the dashboard, navigate to All > Integration Hub > Stream Connect > Stream Connect Home.

The dashboard has the following components.

• Overview section
• Topics
• Consumers
• Message Replications
• Topic Replications

The following sections explain each component in greater detail.

Overview section

View detailed information about your Stream Connect integrations.

Figure 1. Stream Connect Dashboard Overview page

The Overview has these components.

1. Active Insights section

Displays the number of topics, consumers, and topic replicators created.

2. Data Usage Insights section

Displays the following charts.

• Data consumed: The total amount of data consumed (in MB).
• Messages processed: The total number of processed messages.
• Data usage trends: The amount of data used (in KB) every five minutes.

You can view data for a single, selected topic, or for All Topics combined. You can also organize the data by time, viewing data for the Last 24 hours, Last 2 days, Last 7 days, or by setting a Custom range.

Topics

View topics and add, edit, or export a topic from this page.

Figure 2. Stream Connect Dashboard Topics page

The Topics page has the following components.

3. Topics list

Displays all topics and each topic's name, namespace, and the date it was created.

4. Action buttons

Contains several buttons that you can use to perform different tasks. Refresh the list of topics, edit the list columns or reset the column widths, and view or edit the list filters. There are also buttons to Edit a topic, Export a topic, and create a New topic.

• To edit a topic, select the topic from the list, select the Edit button, then fill in the fields on the Edit Kafka Topic form.
• To create a new topic, select New, then fill in the fields on the Create New Kafka Topic form.

You can select a topic from the list to view detailed information for that topic.

Figure 3. Individual topic page

Each individual topic page also has the following components.

5. Topic information section

Displays details about the topic, including its name, namespace, status, number of partitions, date created, domain, and app ID. There’s also an Edit button so you can edit the topic directly from this page.

6. Data Insights

Displays the following charts.

• Messages consumed (per minute): The total number of messages consumed by consumers in the selected topic.
• Consumer lag (per minute): The consumer lag in this topic group, by consumer.
• Bytes consumed (per minute): The total number of bytes consumed.

All three charts can be filtered to display data for the Last 5 minutes, Last hour, or Last 24 hours. When the Last 24 hours is selected, the data is displayed for every five minutes instead of every minute.

7. Consumer (subscriptions) tab

Displays the consumers associated with the topic, and each consumer's number, state, error status, max concurrency, and consumer table. You can select the consumer number to view additional details for that consumer.

8. Replications tab

Displays the topic replications associated with the topic, and each replication's name, topic name, direction, status, whether it's active, and error message. You can select the replication name to view additional details for that topic replication.

Consumers

View all consumers and create new consumers from this page.

Figure 4. Stream Connect Dashboard Consumers page

The Consumers page has the following components.

9. Consumer list

Displays all consumers, organized by type. The possible types are:

• ETL
• Flow Trigger
• Script
• Transform Map

Each consumer is listed with its name, number, state, associated topic, date updated, and who updated it.

10. Create button

To Create a new consumer, select this button, select the type of consumer you want to create, then fill in the fields on the consumer's form. For form field descriptions, see the ETL consumer, Transform Map consumer, or Script consumer page.

Note:

If you're creating a Flow Trigger consumer, there isn't a form to fill out. Instead, the dashboard opens Workflow Studio so you can build your flow trigger there. For a detailed guide, see Create a flow with a Kafka Message trigger.

You can select a consumer from the list to view detailed information for that consumer.

Figure 5. Individual Consumer page

Each individual consumer page has the following components.

11. Consumer information section

Displays details about the consumer, including its number, status, associated topic, Kafka auto position policy, max concurrency, partition group timeout, max batch size, partitions, and an error flag.

12. Overview

Displays the consumer's input and consumption rate, messages processed, queue depth, and any unprocessed messages. It also displays the following charts.

• Messages processed (per minute): The total number of processed messages.
• Message input to topic (per minute): The total number of messages input to the topic.
• Consumer lag (per minute): The lag for the selected consumer.

All three charts can be filtered to display data for the Last 5 minutes, Last hour, or Last 24 hours. When the Last 24 hours is selected, the data is displayed for every five minutes instead of every minute.

13. Statistics tab

Displays consumer statistics, including the date it was created, consumed bytes, consumed messages, produced messages, lag messages, end offset, interval, offset, and partition group.

14. Logs tab

Displays the created logs and each log's level, message, and source.

15. Partition Groups tab

Displays partition groups with each one's number, first and last partition, poll time, number produced per sec, queue depth, and input rate.

For a detailed explanation of consumer data and statistics, including log, and partition group information, see Viewing Kafka subscriptions and statistics.

Message Replications

View message replications, and add, edit, or export a message replication from this page.

Figure 6. Stream Connect Dashboard Message Replications page

The Message Replications page has the following components.

16. Message Replications list

Displays all message replications and each replication's name, messaging service connection, status, error message (if any), and whether it's active.

17. Action buttons

Contains several buttons that you can use to perform different tasks. Refresh the list of message replications, edit the list columns or reset the column widths, and view or edit the list filters. There are also buttons to Edit a message replication, Export a message replication, and create a New message replication.

• To edit a message replication, select the replication from the list, select the Edit button, then fill in the fields on the Edit Message Replication form.
• To create a new message replication, select New, then fill in the fields on the Create New Message Replication form.

You can select a message replication to view the replication's name, messaging service connection, status, error message (if any), and if it's active. You can also see any associated Kafka Topic Replications.

Topic Replications

View topic replications, and add, edit, or export a topic replication from this page.

Figure 7. Stream Connect Dashboard Topic Replications page

The Topic Replications page has the following components.

18. Topic Replications list

Displays all topic replications and each replication's name, direction, replication, topic name, ServiceNow topic, status, error message (if any), whether it's active, and whether it's being tracked in the current update set.

19. Action buttons

Contains several buttons that you can use to refresh the list of topic replications, edit the list columns or reset the column widths, and view or edit the list filters. There are also buttons to Edit a topic replication, Export a topic replication, and create a New topic replication.

• To edit a topic replication, select the replication from the list, select the Edit button, then fill in the fields on the Edit Topic Replication form.
• To create a new topic replication, select New, then fill in the fields on the Create New Topic Replication form.

You can select a topic replication to view its name, direction, replication, topic name, ServiceNow topic, status, error message (if any), if it's active, and if it's being tracked in the current update set. You can also see any associated Message Replication statistics.