Stream connect for Kafka- how to setup customer Kafka instance

Wasif · ‎04-07-2023

Read through about the Stream connect for Kafka that has been introduced in Utah and understood that as opposed to being tied with Confluent Cloud's Rest proxy for Kafka (and accessing the Kafka via Rest) as ServiceNow earlier was in a way a non-native application and connecting to Kafka via Rest was the only option...
and with the new architecture Kafka can be accessed via the Kafka native protocols and is not tied to Rest being used which is an approach for non-native Kafka clients.

We had a kafka requirement in the past, but could not go through coz a) the customer Kafka was not on Confluent cloud with direct Rest proxy enabled b) On AWS or Azure (where their Kafka was present) some additional things needed to be done for the Confluent platform installation for the Rest proxy to be enabled and the cloud team did not see much ROI for development for for just 1 use case for ServiceNow, as other projects/ apps were accessing their Kafka via native methods...

Well coming back to Stream connect,
Based on the architecture, I could not understand how to plug our Kafka environment to ServiceNow.
Some of our non-ServiceNow projects are using Kafka on Azure and some via AWS and we would also like to leverage it and integrate with other apps via Kafka, but not able to understand how to start...has anyone started with this or any POC underway?

Any direction or resource would be helpful.

Attaching screenshot of the new architecture took from the Utah release Creator Toolbox

Wasif · ‎04-09-2023

Any update on this would really help expedite our POC?

amila · ‎04-17-2023

First you need to activate the Hermes plugin[1] or Stream connect plugins as given there. The process of getting plugin activated is same. After that you can access the Hermes through ADCv2 gate way (which is technically a Kafka end point you to connect) as given Here[2]

[1] https://docs.servicenow.com/bundle/utah-servicenow-platform/page/administer/hermes-messaging-service...

[2] https://docs.servicenow.com/bundle/utah-servicenow-platform/page/administer/hermes-messaging-service...

Berlitz Casidsi · ‎04-15-2024

Can Hermes plugin or stream connect be activated on a PDI?

Wasif · ‎06-26-2023

Hi Amila,
Thanks for the reply, however I still need to try the suggestion out.
Will post back once we resume poc work on this. Currently our Kafka poc is stalled.