Use the Kafka Message trigger and Script consumer to consume messages

Retrieve messages from Hermes with the Kafka Message trigger in Workflow Studio and the Script consumer.

Before you begin

• Role required: integration_hub_admin
• This page is part of the Stream Connect quick start guide.
• This document shows you how to consume messages through your connection to Hermes. To set up and test your connection, see Set up a secure connection to the Hermes Messaging Service and Testing the connection to Hermes through the ADCv2 gateway.
• The first part of this page shows you how to use the Kafka Message trigger in Workflow Studio to consume messages. For more information about the trigger, see Create a flow with a Kafka Message trigger. For more information about Workflow Studio and configuring flows, see Workflow Studio.
• The second part of this page shows you how to use the Script consumer to consume messages. For more information, see Configure a script consumer.

Procedure

1. Create a topic to consume messages.
   1. Use the following command to create a topic in Hermes.

      bin/kafka-topics.sh --create --command-config config/bootcamp.properties  --bootstrap-server <instance name>.service-now.com:4000,<instance name>.service-now.com:4001,<instance name>.service-now.com:4002, <instance name>.service-now.com:4003 --topic snc.<instance name>.topic1

      Be sure to replace <instance name> with the name of your instance.
   2. To view the topic, log in to the instance and navigate to All > IntegrationHub > Stream Connect > Topics.
      Note:

      It takes approximately 10 minutes to create a topic.
2. Create a flow trigger to consume messages from the topic.
   1. Navigate to All > Process Automation > Flow Designer.
   2. Select Create new > Flow.
   3. On the Flow properties form, enter a name for your flow, then enter Submit.
      This example uses the name Topic1 Consumer.
   4. In the trigger section, select Add a trigger, then select Kafka Message as the trigger type.

      

   5. Select the Topic1 topic.

      

      You can use the magnifying glass icon to see the list of available topics.

   6. In the Actions section of the flow, navigate to Add an Action, Flow Logic, or Subflow > Flow Logic > For Each.
   7. Drag the Messages data pill to the Items field, then select Done.
   8. In the Actions section of the flow, navigate to Add an Action, Flow Logic, or Subflow > Action > ServiceNow Core > Log.
   9. Drag the Payload data pill to the Message field, then select Done.

      

   10. Save the flow trigger.
   11. Go to the Settings [sys_flow_execution_setting] table to enable reporting for this flow.
   12. Create a new entry and enable reporting to the flow created above.

       

   13. Navigate back to your flow and select Activate.
   14. To view your topic subscription, navigate to All > Integration Hub > Stream Connect > Subscriptions.

       

3. Publish messages to the topic using the Kafka command-line tool.
   1. Run the following command to publish messages to the topic.

      bin/kafka-console-producer.sh --topic snc.<instance name>.topic1  --producer.config  config/bootcamp.properties  --bootstrap-server <instance name>.service-now.com:4000,<instance name>.service-now.com:4001,<instance name>.service-now.com:4002,<instance name>.service-now.com:4003

      Be sure to replace <instance name> with the name of your instance.

      This command prompts you for messages. Add a few sample messages. This example uses the following messages.

      • Test message1
      • Test message2
      • Test message3

      You should receive the messages in about a minute.
   2. Navigate back to the flow and select Executions.

      

   3. Open the flow execution.

      

   4. To view the statistics for this subscription, navigate to All > Integration Hub > Stream Connect > Subscriptions, open the subscription, and go to the statistics view.

      

4. Create a Script consumer to consume messages from the topic.
   1. Navigate to IntegrationHub > Consumers > Script Consumers.
   2. Select New.
   3. In the Name field, enter a name for your script consumer.
      This example uses Topic1 Consumer.
   4. Set the script to log the message by adding the following logic in the script box.

      for (var i = 0; i < messages.length; i++) {
            gs.info('Received message ' + JSON.stringify(messages[i].message));
       }
      

      

   5. Save the form.
   6. In the Related Links section, select New to add a Kafka stream to consume the messages.
   7. On the Kafka Stream form, set the following field values.
      • Name to Topic1 Stream.
      • Topic to the topic created earlier, topic1.
      • Max concurrency to 1.
      • Start Consuming to All the messages stored in the topic. This option enables you to read the messages already published to this topic.
      • Message handling to Automatically optimize.

      

   8. Select the Activate link under Related Links.
      You should be able to see the activated subscription.
   9. To view the messages, navigate to All > System logs > System log and filter messages that start with Received message.

      

   10. You should see all the messages after 1 minute.

What to do next

Use the ETL and Transform Map consumers to import data.