Kafka stream connect - message replication error

karthikeyan R
Tera Contributor

‎08-16-2024 12:46 AM

Hi,

 

I am trying to post a topic from local kafka to ServiceNow Kafka using message replication . getting below error.  Any recommendation for the fix?

 

Mid server agent logs :

 

2024-08-16T03:03:04.644-0400 ERROR (kafka-adapter-2f6096bc3bbb865091553364c3e45a2d-worker0) [StreamConnectLogger:74] Stream Connect - KafkaDestinationService : Delivery to snc.xxgntsandbox1.sn_streamconnect.xx_mongo_topic failed with TimeoutException: Topic snc.xxgntsandbox1.sn_streamconnect.xx_mongo_topic not present in metadata after 60000 ms

 

 

  • 557 Views
1 REPLY 1

Liju John1
Mega Guru

‎06-09-2025 02:30 PM - edited ‎06-10-2025 09:17 AM

Add the following json to the kafka_connection_list.do-> Additional Client Properties.

 

ISSUE: Stream Connect Message Replication Error.
"Delivery to snow-vr-cmdbci failed: Expiring 120 record(s) for snow-vr-cmdbci-1:120002 ms has passed since batch creation". 
See Kafka Topic Replications: https://xxx.service-now.com/sys_kafka_topic_replication_list.do

Also, check the broker health in the local kafka env.

 

Kafka_connection-Additional_properties.png

{
"request.timeout.ms": "30000", // 30s: how long to wait for a broker response
"linger.ms": "100", // 100ms: buffer messages a bit to enable batching
"batch.size": "262144", // 256KB: good for reducing the number of requests
"buffer.memory": "134217728", // 128MB: memory pool for message batching
"delivery.timeout.ms": "300000", // 5min: max time for message delivery attempts
"acks": "all", // strongest delivery guarantee (wait for all ISR)
"retries": "5", // retry failed sends up to 5 times
"compression.type": "gzip" // compression will significantly reduce the size of the messages
}

 

0 Helpfuls