--- sourceDocument: Australia Workflow Data Fabric sourceDocumentLink: https://www.servicenow.com/docs/r/australia/integrate-applications Release : - australia ft:locale : - en-US ft:publication_title : - Australia Workflow Data Fabric ft:clusterId : - crint bundleId : - crint workflow : - Creator --- # Message replication # Stream Connect Message Replication {#ariaid-title1} Release version: Australia Updated March 12, 2026 ![](https://www.servicenow.com/docs/portal-asset/ico-clock) 3 minutes to read Summarize ![AI sparkle icon](https://servicenow.com/docs/portal-asset/ai-sparkle-icon) Summarized using AI This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete. ## Summary of Stream Connect Message Replication Stream Connect Message Replication enables ServiceNow customers to replicate data seamlessly between their local Apache Kafka environments and ServiceNow instances. This capability is managed directly within ServiceNow and leverages a MID Server or MID Server cluster to execute data replications without the need for additional external services. The solution simplifies setup by automatically generating the necessary certificates for secure communication. Show full answer Show less ## Configuring MID Servers The replication process uses a MID Server with a replicator extension that must have network access to Hermes endpoints through specific firewall port ranges (Producer: 4000--4050, Consumer1: 4100--4150, Consumer2: 4200--4250). It's important to note that MID Server proxy settings do not apply to Hermes connectivity, as Hermes uses Kafka-native TCP connections. Customers can enhance reliability and load distribution by deploying a MID Server cluster with the Load Balance cluster type (Failover clusters are not supported). The cluster automatically redistributes topic partitions if MID Servers become unavailable or new ones are added. ## Connections and Credentials To connect to Kafka, configure Kafka credentials and connections within ServiceNow, then create a Connection \& Credential alias that associates these details. This alias must be accessible by the MID Server. No manual credential setup is needed for Hermes, as the MID Server manages the required keystore, truststore, and certificates automatically. ## Creating and Managing Replications Replication setup involves two key record types: * **Message Replication records:** Represent Kafka clusters and act as parents for topic replications. Each Kafka cluster requires its own Message Replication record. * **Kafka Topic Replication records:** Define replication from a single source topic to one destination topic. One-to-many replications from a single source topic are not supported. These records reside in dedicated tables within ServiceNow and can be created and managed through guided procedures. ## Monitoring and Logging Once replication is active, metrics are generated every 60 seconds per topic replication, including message counts and other relevant data. These metrics are accessible via the Message Replication Statistics table or directly on individual Kafka Topic Replication records, allowing customers to monitor replication health and performance. Most replication issues are logged in the MID Server log. For deeper troubleshooting, debug logging can be enabled by setting a specific MID Server property. ## Access and Permissions To manage replications, users require the **messagereplicationadmin** role, which grants full create, modify, and delete permissions across replication and connection tables. Users with the **messagereplicationuser** role have read-only access to view replication records. ## Requirements This functionality requires activation of the **ServiceNow Stream Connect Installer \[com.glide.hub.streamconnect.installer\]** plugin. Replicate data between your Apache Kafka environment and ServiceNow. With Stream Connect Message Replication, you can configure and manage message replications directly from your ServiceNow instance. Stream Connect Message Replication uses a MID Server or MID Server cluster to run the data replications, so you don't need to configure or host additional replication services. It also simplifies the message replication setup by automatically generating the required certificates. ## Enabling a MID Server to replicate data {#stream-connect-message-replication__section_fg5_qch_nzb} Stream Connect Message Replication uses a MID Server with a replicator extension to replicate data to and from your local Kafka. For instructions on how to configure the MID Server, see [Configuring MID Servers](https://www.servicenow.com/docs/access?context=c_MIDServerConfiguration&version=australia&pubname=australia-servicenow-platform&ft:locale=en-US). The MID Server must be able to access Hermes endpoints through the firewall. Work with your network administrator to make sure that the following port ranges are open. * Producer: 4000--4050 * Consumer1: 4100--4150 * Consumer2: 4200--4250 {#stream-connect-message-replication__ul_lzv_pj5_pzb} Note: Proxy settings on the MID Server don't apply to Hermes connectivity. The MID Server proxy configuration applies only to HTTP-based communication. Hermes uses Kafka-native TCP connections, so traffic to the Hermes cluster bypasses proxy settings regardless of how the MID Server is configured. ## Using a MID Server cluster {#stream-connect-message-replication__section_dxx_k14_nfc} You can use a MID Server cluster, instead of a single MID Server, for message replication. With a MID Server cluster, if one of the MID Servers in the cluster fails, the other MID Servers can share the load of the failed MID Server. In a MID Server cluster, the topic partitions are distributed across all the available MID Servers in the cluster. If a MID Server becomes unavailable, the partitions are redistributed on the remaining MID Servers. If an additional MID Server becomes available in the cluster, then the partitions are distributed across the MID Servers again. For details on setting up a MID Server cluster, see [Configure a MID Server cluster](https://www.servicenow.com/docs/access?context=t_ConfigureAMIDServerCluster&version=australia&pubname=australia-servicenow-platform&ft:locale=en-US). Note: The MID Server cluster must have the Load Balance cluster type. The Failover cluster type is not supported. ## Configuring connections and credentials for Kafka {#stream-connect-message-replication__section_frm_3ch_nzb} Stream Connect Message Replication uses a Connection \& Credential alias to connect to your local Kafka. To configure a Connection \& Credential alias: 1. [Create Kafka credentials](https://www.servicenow.com/docs/NnI~A0_WGOUF~e1CTvbl~A "Configure credentials for your Apache Kafka connection.") with the authentication data required for the connection. 2. [Configure a Kafka connection](https://www.servicenow.com/docs/nhfcddHhhBIEpwcHyA023Q "Configure a connection to your Apache Kafka environment.") to connect to your Kafka environment. 3. [Create a Connection \& Credential alias](https://www.servicenow.com/docs/access?context=connection-alias&version=australia&pubname=australia-platform-security&ft:locale=en-US), to associate the connection information and credential data. The Connection \& Credential alias should have a Connection type of Kafka and must be accessible from the MID Server. {#stream-connect-message-replication__ol_x4t_jtb_qzb} These steps are for configuring a Connection \& Credential alias for connecting to Kafka. You don't need to set up connections or credentials for connecting to Hermes, because the MID Server automatically handles the creation of the required keystore, truststore, and certificates. ## Creating message and topic replications {#stream-connect-message-replication__section_qkq_y1h_nzb} Message replication requires Message Replication records and Kafka Topic Replication records. A Message Replication record represents a single Kafka cluster. For example, if you have two Kafka clusters, you would create two different Message Replication records, one for each cluster. A Message Replication record is the parent record for all the topics being replicated to or from that cluster. Message Replication records are stored in the Message Replications \[sys_sc_message_replication\] table. A Kafka Topic Replication record specifies the replication from a single source topic to a single destination topic. You can't replicate a single source topic to multiple destinations. You can only replicate to each destination once. Kafka Topic Replication records are stored in the Kafka Topic Replications \[sys_kafka_topic_replication\] table. For a step-by-step guide to creating message and topic replication records, see [Create message and Kafka topic replications in Stream Connect](https://www.servicenow.com/docs/mqRzu8qZM8XZx_oLn0bbgA "Set up message and topic replications to replicate data between your Apache Kafka environment and ServiceNow."). ## Viewing message replication statistics {#stream-connect-message-replication__section_apc_hch_nzb} Once replications are running, the system creates a metrics record for each active topic replication every 60 seconds. Metrics records provide information about topic replications, including the Message count, which shows the number of messages replicated in each collection interval. You can view metrics records on the Message Replication Statistics \[sys_sc_channel_replication_metric\] table. You can also view metrics records for a particular topic by checking the Message Replication Statistics on its Kafka Topic Replication record. For a list of message replication metrics and their descriptions, see [Viewing Stream Connect Message Replication statistics](https://www.servicenow.com/docs/pnVQQwLk6jjCR3szvTXGfw "When message replications are running, the system creates a metrics record for each active topic replication every 60 seconds. Metrics records provide information about topic replications. You can view metrics records on the Message Replication Statistics [sys_sc_channel_replication_metric] table. You can also view metrics records for a topic by checking the Message Replication Statistics on its Kafka Topic Replication record."). ## Required plugin {#stream-connect-message-replication__section_isz_51h_nzb} Stream Connect Message Replication requires the ServiceNow Stream Connect Installer \[com.glide.hub.stream_connect.installer\] plugin. ## Roles {#stream-connect-message-replication__section_qqr_5xg_nzb} The message_replication_admin role can create, modify, and delete records in all the message replication tables, including the connection and credential tables, and the message and topic replication tables. The message_replication_user role can view records in the message replication tables. ## Logging {#stream-connect-message-replication__section_jgl_gsz_mzb} Most issues are logged in the MID Server log. Additional debug logging can be enabled by setting the glide.stream_connect.message_replication.debug MID Server property to true. * **[Create message and Kafka topic replications in Stream Connect](https://www.servicenow.com/docs/mqRzu8qZM8XZx_oLn0bbgA)** Set up message and topic replications to replicate data between your Apache Kafka environment and ServiceNow. * **[Create Kafka credentials](https://www.servicenow.com/docs/NnI~A0_WGOUF~e1CTvbl~A)** Configure credentials for your Apache Kafka connection. * **[Configure a Kafka connection](https://www.servicenow.com/docs/nhfcddHhhBIEpwcHyA023Q)** Configure a connection to your Apache Kafka environment. * **[Viewing Stream Connect Message Replication statistics](https://www.servicenow.com/docs/pnVQQwLk6jjCR3szvTXGfw)** When message replications are running, the system creates a metrics record for each active topic replication every 60 seconds. Metrics records provide information about topic replications. You can view metrics records on the Message Replication Statistics \[sys_sc_channel_replication_metric\] table. You can also view metrics records for a topic by checking the Message Replication Statistics on its Kafka Topic Replication record.