Local Kafka to Service Now connectivity issue

[ramprasad1910]

Hi All,

I am trying to establish connectivity between Strimzi Kafka (local Kafka instance) and Service Now Hermes Messaging service. I have created Hermes Topic and configured the necessary SSL credentials in Service Now along with the Message Replication. However, I am still not able to establish connectivity.

I checked the Mid server logs regarding the Service Now to Kafka connectivity and we saw the below error message. Kindly update this information in our Service Now case.

2024-08-01 09:01:09 ERROR (kafka-producer-network-thread | producer-4) [DebugLogger:36] Stream Connect Message Replication - HermesSslEngineFactory : Error creating Hermes Context
java.lang.UnsupportedOperationException: Config is shutdown
at com.service_now.mid.services.Config$ShutdownConfigManager.getSetting(Config.java:605)
at com.service_now.mid.services.Config.getProperty(Config.java:204)
at com.service_now.mid.message_replication.hermes.HermesSslEngineFactory.getProtocol(HermesSslEngineFactory.java:121)
at com.service_now.mid.message_replication.hermes.HermesSslEngineFactory.createHermesContext(HermesSslEngineFactory.java:89)
at com.service_now.mid.message_replication.hermes.HermesSslEngineFactory.createClientSslEngine(HermesSslEngineFactory.java:42)
at org.apache.kafka.common.security.ssl.SslFactory.createSslEngine(SslFactory.java:203)
at org.apache.kafka.common.security.ssl.SslFactory.createSslEngine(SslFactory.java:189)
at org.apache.kafka.common.network.SslChannelBuilder.buildTransportLayer(SslChannelBuilder.java:122)
at org.apache.kafka.common.network.SslChannelBuilder.buildChannel(SslChannelBuilder.java:105)
at org.apache.kafka.common.network.Selector.buildAndAttachKafkaChannel(Selector.java:338)
at org.apache.kafka.common.network.Selector.registerChannel(Selector.java:329)
at org.apache.kafka.common.network.Selector.connect(Selector.java:256)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:1032)
at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:73)
at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1203)
at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1091)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:569)
at org.apache.kafka.clients.NetworkClientUtils.isReady(NetworkClientUtils.java:42)
at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:64)
at org.apache.kafka.clients.producer.internals.Sender.awaitNodeReady(Sender.java:564)
at org.apache.kafka.clients.producer.internals.Sender.maybeSendAndPollTransactionalRequest(Sender.java:485)
at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:332)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:247)
at java.base/java.lang.Thread.run(Thread.java:833)

Kindly let me know if I can get any assistance in this regard to resolve this issue.

[Rajesh Chopade1]

Hi @ramprasad1910 

The error message you're encountering suggests that there is an issue with the configuration or context of the Kafka producer, specifically related to the SSL/TLS setup. The java.lang.UnsupportedOperationException: Config is shutdown indicates that the configuration object is in a state where it cannot be used, possibly because it has been closed or improperly initialized.

Verify the SSL configuration parameters in your Kafka producer properties. Ensure all required SSL properties are correctly set.

i hope my answer helps you to resolve your issue, if yes mark my answer helpful & correct.

THANK YOU

rajesh chopade

[ramprasad1910]

Thank you Mr. Rajesh Chopade for your reply. I have a couple of questions to clarify.

1. Does Service Now offer support with Strimzi Kafka?

2. Does Stream connect work without an SSL?

[Rajesh Chopade1]

Hi @ramprasad1910 

ServiceNow does not natively offer built-in support or direct integration for Strimzi Kafka. Strimzi is a Kubernetes operator that simplifies the deployment and management of Apache Kafka on Kubernetes, and it operates independently of ServiceNow.

However, you can integrate ServiceNow with Kafka (including a Kafka instance managed by Strimzi) using various methods:

• Custom Middleware: You can write custom middleware that interacts with Kafka and uses ServiceNow's REST API to communicate with ServiceNow.
• ServiceNow IntegrationHub: You might need to use IntegrationHub custom actions if direct Kafka integration is required.
• Third-party Middleware: Middleware solutions that can bridge between ServiceNow and Kafka.

Kafka without SSL

Yes, Kafka Connect can work without SSL. SSL (Secure Sockets Layer) is used to encrypt data transmission to ensure secure communication between Kafka brokers and clients. However, in development or internal environments where security is not a concern, you can configure Kafka Connect to work without SSL.

Here’s an example configuration for a Kafka connector without SSL:

{
    "name": "example-connector",
    "config": {
        "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
        "tasks.max": "1",
        "connection.url": "jdbc:mysql://your.database.server:3306/yourdatabase",
        "connection.user": "yourusername",
        "connection.password": "yourpassword",
        "table.whitelist": "yourtable",
        "mode": "incrementing",
        "incrementing.column.name": "id",
        "topic.prefix": "your-topic-prefix-",
        "key.converter": "org.apache.kafka.connect.json.JsonConverter",
        "value.converter": "org.apache.kafka.connect.json.JsonConverter",
        "key.converter.schemas.enable": "false",
        "value.converter.schemas.enable": "false"
    }
}

I hope my answer helps you to resolve your issue, if yes mark my answer helpful & correct.

THANK YOU

rajesh chopade

[LakshmiSraK]

Hi @ramprasad1910 

Please follow the below steps to successfully stream to kafka to fix the SSL handshake issue. 

1. Create Topic in Hermes
2. Create Credential&Connection Alias in Servicenow with type as Kakfa
3. Get the Certificate details of your broker by running below command from your local terminal

oenssl s_client -connect <brokername> -showcerts

4.  Save it into a .pem file. From begin certificate to end of certificate

5. Convert .pem to .jks format

keytool -importcert \

  -alias KafkaTrust \

  -file <.pem filename> \

  -keystore <.jks filename> \

  -storepass <set a proper password> \

  -noprompt

6.Convert to base64

cat <.jks file name> |base64     

7. Save the output into .txt file

8. Create kafka SSl credentials in Servicenow and paste the ssl truststore value from .txt file

9. Create connection and tag that kafka SSL credential . Use your required midserver for the connection

10. Create Message replication from servicenow and tag the proper kafka topic created in Servicenow with the topic form your consumer

11. Create Flow designer for producing or consuming messages

https://www.youtube.com/watch?v=rCj0t4iAJRw&t=2562s