Configuring the ThousandEyes Connecter for Event Management.

[Mike I2]

Hi all, 

I'm trying to connect ThousandEyes to my client's instance. I've worked through SN and ThousandEyes doc's pages, and can successfully retrieve a token, however whilst trying to test the webhook in ThousandEyes by sending a test event, we're getting a 403 Forbidden error with just the following log entry immediately after each test attempt. Has anyone had a similar issue?

09/08/2023 13:58:32

Error

DxC_ML: look up window size = 0, it should be positive: no thrown error

com.glide.ui.ServletErrorListener

[Mike I2 · Accepted solution]

After much trial and error, I've got this working. The main issue was incomplete and incorrect @Servicenow documentation. The following steps are needed to fully complete this integration:

ServiceNow Configuration

1. Create a new HTTP(S) Endpoint CI record with the URL "https://<instance_name>.service-now.com/api/sn_em_connector/em/inbound_event?source=thousandeyes"
2. Create a service account for ThousandEyes in the sys_user table with the roles web_service_admin, and evt_mgmt_integration
3. Create an Oauth Configuration for the ThousandEyes Push Connector
   1. Navigate to All > System OAuth > Application Registry and then select New.
   2. Select Create an OAuth API endpoint for external clients.
   3. Add the following Redirect URLs as a comma-separated list
      1. https://app.thousandeyes.com/webhooks-oauth-callback/
      2. https://app.thousandeyes.com/namespace/integrations/AuthCallbackPage.html

ThousandEyes Configuration

1. Create an Integration: Click Integrations in the left side bar menu, then the New Integration button.
2. Complete the form as follows:
   1. Name -  can be anything
   2. URL = Endpoint URL from the HTTP(S) Endpoint CI record, e.g "https://<instance_name>.service-now.com/api/sn_em_connector/em/inbound_event?source=thousandeyes"
   3. Preset Configurations = Generic
   4. Auth Type = OAuth Code
      1. Auth URL = https://<instance_name>.service-now.com/oauth_auth.do
      2. Token URL = https://<instance_name>.service-now.com/oauth_token.do
      3. Client ID = copied from Oauth Configuration
      4. Client Secret = copied from Oauth Configuration
   5. Click Get Token. This will open a ServiceNow login page. Enter the credential for the service account created in a previous step. 
   6. Click Login, then click Allow
   7. You will be redirected back to the previous page. Click Save to save the custom webhook configuration
   8. Check in ServiceNow logs for a log entry confirmation:
      1. source = OAUTH_SERVER
      2. level = Information
      3. message similar to "OAuth token(s) of access_token refresh_token is/are granted to app:ThousandEyes;grant_type:authorization_code;user:thousandeyes"
   9. Back in ThousandEyes Integration window, click the custom webhook, status = Untested, to open it.
   10. Click Test
   11. Wait for the following message
   12. 
   13. Click Save again Webhook status should now be Connected. The integration is now complete 
   14. Create a test - Alerts > Alert Rules in the left hand side bar menu. 
   15. Select any rule
   16. 
   17. Click Notification. 
   18. Under Webhooks, click Edit webhooks. 
   19. Click Add New Webhook
       1. Auth Type = Oauth
       2. Name = can be anything
       3. URL = Endpoint URL from the HTTP(S) Endpoint CI record, e.g "https://<instance_name>.service-now.com/api/sn_em_connector/em/inbound_event?source=thousandeyes"
       4. Grant Type = Implicit
       5. Auth URL = https://<instance_name>.service-now.com/oauth_auth.do
       6. Client ID = copied from Oauth Configuration
       7. Click Get Token. This will open a ServiceNow login page. Enter the credential for the service account created in a previous step. 
       8. Click Login, then click Allow
       9. You will be redirected back to previous ThousandEyes window. Wait for "Token retrieved successfully" confirmation
       10. Click Test. Wait for "Webhook test completed successfully" confirmation.
       11. Click Save Webhook. 
       12. This will have created a test Event in Servicenow:
       13. Open Servicenow. Open All Events. Look for an event with event source = ThousandEyes, Description = WEBHOOK_TEST, e.g 
       14. 
   20. The configuration is now complete. The integration and webhook are now connected to your ServiceNow instance, and Event Management can now be configured to process incoming events from ThousandEyes. 

[Prashant62]

Hi Mike,

I have also configured the ServiceNow integration with ThousandEyes using custom webhook with the help of steps mentioned by you. We have also received the webhook test event but we are not receiving the events from ThousandEyes even after that. We have added the webhook to the Notifications section in ThousandEyes. Could you please help us where we can check the logs or troubleshoot on this issue.

Thanks,
Prashant Gupta

[MortenPettersen]

Hi Prashant

Activating more logging for the sn_em_connector API might help in understanding where it goes wrong. If the Events are received by the API you can have a look at the incoming payloads.

System Property "global.cloud.event.debug" can be created and set to true.

Then you should be able to look at raw data that is coming into the API by checking the records that will be created in the "Push Connector Debugs" table [sn_em_connector_listener_debug].