Exploring Hermes Messaging Service
Summarize
Summary of Exploring Hermes Messaging Service
The Hermes Messaging Service enables seamless integration between your Apache Kafka environment and your ServiceNow® instance for efficient data exchange. Built on Apache Kafka, it is a multi-tenant, multi-cluster data transport and queuing service that supports producing and consuming large volumes of Kafka events with low latency. Hermes simplifies asynchronous data transport, manages performance during message spikes, and provides diagnostic tools to monitor message delivery health.
Show less
Key Features
- Kafka Integration: Enables your instance to publish events to and consume events from Kafka topics efficiently.
- Scalability: Supports partitioned topics for scalable and parallel event processing, ensuring ordered processing within partitions.
- Namespace Management: Organizes topics logically by namespace, which can represent Kafka clusters or domains, allowing access control in domain-separated instances.
- Monitoring and Administration: Hermes administrators can create and manage namespaces and topics, and inspect message payloads using the topic inspector.
- Support for Multiple Use Cases: Integrates with ServiceNow capabilities such as Stream Connect, Log Export Service (LES), and Instance Data Replication (IDR).
How Hermes Works
- Events are key-value records representing data changes or occurrences.
- Events are grouped into topics, which are divided into partitions for scalability.
- Producers send events to topics, assigning keys that determine partition placement.
- Consumers read and process events from topic partitions in order.
- Hermes administrators create and manage namespaces and topics and monitor message flow.
Benefits for ServiceNow Customers
- Flexible Data Integration: Various ServiceNow applications can publish and consume Kafka events asynchronously.
- Enhanced Platform Capabilities: Hermes is integrated with AI Platform features and supports workflows, log exports, and data replication.
- Improved Efficiency: Handles high-volume message exchanges with low latency and manages bursts to protect instance performance.
Practical Limits
- Maximum message size: 2 MB per message and per second for both producing and consuming.
- Supports up to 30 topics per instance and 32 partitions per topic.
- Message retention is limited to 36 hours.
- Supported compression types: GZIP, LZ4, and NONE for producing messages.
Next Steps for Customers
To utilize Hermes Messaging Service effectively, customers should explore configuration guides, data exchange methods, namespace and topic management, and administrative tools provided by ServiceNow. These resources will help optimize integration scenarios and ensure smooth Kafka event processing within your ServiceNow instance.
Learn about the Hermes Messaging Service and how it helps you manage the flow of data between your Kafka environment and your instance.
Hermes Messaging Service overview
Apache Kafka is an open-source data streaming platform that provides a single integration point for exchanging data across business systems in your organization. The Hermes Messaging Service enables you to integrate your Apache Kafka environment with your ServiceNow® instance to exchange data with these business systems.
The Hermes Messaging Service is a multi-tenant, multi-cluster, data transport, and queuing service built on Apache Kafka that enables your instance to produce and consume large volumes of Kafka events.
- Simplify and standardize asynchronous data transport between your Kafka environment and your ServiceNow instance.
- Publish and process Kafka events at scale. Publish events to your Kafka environment from your ServiceNow instance and consume Kafka events from your external systems at a high volume with low latency.
- Effectively process spikes and bursts in message volume to minimize performance impacts on your instance.
- Manage the health and performance of message delivery using Hermes diagnostic tools.
Components
Hermes uses the following components.
- Event
- A record of something that happened, such as the creation of a log entry. Kafka uses events to transmit data between systems in near real time. An event includes a key and a value.
- Topic
- A logical group of events. In Kafka, these events are transmitted as messages. Topics are used to group related events together. Events in a topic are organized using partitions.
- Partition
- A topic is divided into partitions to support scalability and parallel processing. Each event is assigned to a partition based on its key. Events with the same key are always written to the same partition in order of arrival.
- Namespace
- A logical grouping of topics. You can organize topics by namespace to indicate which Kafka cluster they come from. In a domain-separated instance, you can create a namespace for each domain to control access to specific topics.
- Hermes Kafka cluster
- A Kafka cluster hosted in a ServiceNow data center. Hermes uses Kafka clusters to manage the delivery of messages between producers and consumers.
- Producer
- A system or component that creates and sends events to a topic in the Hermes Kafka cluster. The producer sets the event key, which determines the partition where the event is stored.
- Consumer
- A system or component that reads events from one or more topic partitions. Consumers process events in the order they appear within each partition. Consumers can read from multiple partitions simultaneously.
Hermes Messaging Service users
| User | Description |
|---|---|
| Hermes administrator | The Hermes admin defines namespaces for each Kafka cluster or domain and creates topic records in the instance or through the command-line interface (CLI). The Hermes admin monitors topics and views the payload of a message in a Hermes topic using the Hermes Messaging Service topic inspector. |
Hermes Messaging Service workflow
- In a domain-separated instance, the Hermes admin creates namespace records for different domains.
- The Hermes admin creates topic records in the instance or through the CLI.
- The Hermes admin updates or rescans topic records as needed in the instance.
- The Hermes admin monitors topics and views message payloads using the Hermes Messaging Service topic inspector.
Benefits
Different integration points and applications in the ServiceNow platform can use the Hermes Messaging Service to publish and ingest events.
The Hermes Messaging Service is a ServiceNow AI Platform capability that is available as part of Stream Connect, Log Export Service (LES), and Instance Data Replication (IDR).
| Benefit | Application | Users |
|---|---|---|
| Export data from your ServiceNow instance to an external application using the Hermes Messaging Service with Workflow Studio. | Flow Designer | Administrator |
| Export log data securely from your instance to your external data lake for reporting, auditing, or regulatory purposes by using the Hermes Messaging Service with the Log Export Service. | Log Export Service (LES) | Administrator |
| Trigger flows from incoming Kafka events and trigger outgoing messages to your Kafka environment using the Hermes Messaging Service with Stream Connect. | Using Stream Connect for Apache Kafka | Administrator |
| Improve the performance and processing efficiency of Instance Data Replication (IDR) by upgrading replication sets that you created prior to the Utah release to use the Hermes Messaging Service. | Instance Data Replication | Administrator |
Limitations
- Maximum message size: 2 MB
- Maximum message size per second: 2 MB (produced or consumed)
- Number of topics: 30 topics per instance
- Maximum number of partitions: 32 partitions per topic
- Retention policy: 36 hours
- Supported compression types for producing messages to Hermes: GZIP, LZ4, and NONE.