Scripted REST API Webhooks and the Replicate AI Integration in CreatorCon C3

[Travis Toulson]

Check out other articles in the C3 Series

What part of CreatorCon C3 drove me to screaming into a pillow... repeatedly? Honestly, I'm surprised that no one has asked me this question yet, but that honor goes to the Replicate AI integration. And keep in mind, there is nothing inherently wrong with the API, it's fantastic and well documented. But imagine you are diving into webhooks for the first time, using unauthenticated Scripted REST API HMAC validation for the first time, diving into ServiceNow crypto API's for the first time, and every time you test the integration you have to wait around 2 minutes for a response to see if you were successful.

Screaming. Into. A. Pillow.

But through the process, I learned a number of valuable things about webhooks, using credentials in ServiceNow, and orchestrating asynchronous request-response based integrations on ServiceNow. In this article, let's dive into the technical aspects of the Replicate AI integration so you can learn from my experiences.

The Integration Process

First, let's look at the dataflow through the different components and scripts on the ServiceNow side of the integration:

1. Generated Image record state updates to Pending Image Generation
2. Business Rule queues the AI Image Requested Event
3. A Script Action handles the event by sending a POST to Replicate AI with:
   1. The Base64 string representation of the player's submitted photo
   2. The avatar creation prompt including options the player submitted such as gender and hair color
   3. A URL for the Avatar Created webhook (Scripted REST API) that Replicate should call once the avatar is created
4. Replicate AI runs a model to generate the avatar and upon completion calls the Avatar Created webhook URL with:
   1. The URL of the avatar image
5. The Avatar Created Scripted REST API will then
   1. Send a GET to retrieve the image file from the URL
   2. Save the image to the Generated Image record
   3. Queue the Background Remove Requested Event
6. A Script Action handles the event by sending a POST to Replicate AI with:
   1. The URL of the avatar image provided by Replicate AI
   2. A URL for the Background Removed webhook (Scripted REST API) that Replicate should call once the avatar has been isolated from the background of the image
7. Replicate AI runs a model to remove the background from the avatar image and upon completion calls the Background Removed webhook URL with:
   1. The URL of the avatar image with background removed
8. The Background Removed Scripted REST API will then
   1. Send a GET to retrieve the image file from the URL
   2. Save the image to the Generated Image record
   3. Update the state of the Generated Image record to Pending Selection

This process at first glance may seem to be a little more complex than it needs to be. Why doesn't Replicate just send the image Base64 representation in the original POST response? Why is C3 firing an event instead of just making the REST call in the Business Rule? The whole process would be so much simpler if everything could be done in the Business Rule and it seems like it should be possible.

So why isn't it?

Webhooks and long running processes

It seems like the whole avatar creation process should be able to happen in one or at most two REST calls. In this case though, it's taking six REST calls in total. Let's look at the REST calls involved in this integration.

REST Calls in C3 - Replicate AI Integration:

1. C3 POST to Replicate AI to queue a prediction to generate an avatar
2. (Webhook) Replicate AI POST the resulting avatar URL to C3
3. C3 GET the avatar URL to obtain the image file
4. C3 POST to Replicate AI to queue a prediction to remove the background from the avatar image
5. (Webhook) Replicate AI POST the resulting avatar with background removed URL to C3
6. C3 GET the avatar URL to obtain the image file

So why all these POST and POST backs? Because Replicate AI is running AI models which are typically long running processes. It can sometimes take up to 5 minutes to generate an avatar. And because the resources running the AI model were limited, the prediction requests were put in a queue. At one point during testing, some avatars were taking 20 minutes to get through the queue!

How do you feel when you call a support number with a question and have to wait 20 minutes for an answer? Well, servers don't feel too great about it either. And in the same way that some support numbers let you request a callback when you reach the front of the line... Webhooks let you give a server a URL that they can call when something happens.

In this case, C3 asks Replicate AI to run some AI Model and gives it a URL that should be called when Replicate AI is done. Then C3 isn't tying up its server resources waiting for an answer.

Why are the GET requests necessary? The simple answer is that JSON doesn't support a binary data type that image files are stored in. We'll tackle more on binary files in another article.

Credentials aren't visible to most users

So, why use ServiceNow's event queue? Why not just throw these calls directly into the Business Rule and the Scripted REST API?

Well, as it turns out, most ServiceNow users are unable to view Credential records in ServiceNow. You know, the records an integration uses to authenticate one server to another. So, when our script made a call to retrieve the credential (slightly modified from its original object-oriented style):

let token;

function getAPIToken() {
    if (token) {
        return;
    }

    const credentialAlias = 'sys_id_of_credential_alias';
    const provider = new sn_cc.StandardCredentialsProvider();
    const cred = provider.getCredentialByAliasID(credentialAlias);
    token = cred.getAttribute('api_key');
}

The script failed to find a credential and C3 was unable to authenticate with Replicate AI. This left us with a few options:

1. Alter the ACL's of the Credential table to allow all users to see at least the credential they need access to
2. Store the credential in a property that anyone can see
3. Run the scripts as a user who CAN see the credentials

If my strikethrough text isn't clear enough, I think the first two are terrible ideas. I want users to be able to generate avatars THROUGH C3 and all the security rules baked in. Anything that allows the end user to see the Replicate AI authentication token would allow them to run models in Replicate AI directly without having to go through C3. And sure, you would have to know to look for it but that's not a risk I want to take.

In my mind, the best solution is running the specific scripts in the process as a system user. This is exactly how ServiceNow Events and Script Actions work. A better alternative, if I was a little more Flow Designer savvy, would have been to use a Flow / Integration Hub to handle the integration. Flow Designer let's you easily specify Flows that should run as System or even as a specific user. But I'm an old dog struggling to learn new tricks... so Event registry won.

On Traceability and Orchestrating Async Processes

Some of my astute readers might suggest that I could have solved the previous issue with Async Business rules. If I recall correctly, I do believe those run as system user as well. So why events? Why an extra Event Registry record and extra Script Action?

Traceability.

When working with highly asynchronous processes, do not underestimate the importance of traceability! With synchronous processes, you can easily throw a script debugger at it and follow it from start to finish. Async processes can be a nightmare to troubleshoot without good end to end traceability. Early in the development, we had a number of steps in the card creation firing out of order due to some missteps in the triggers.

From this, we created a dedicated logging table for the integration where we could associate logs with the records of our data model, committed to using things like Events that record their own execution, and implemented states for darn near everything when it came to Photo Submissions and Generated Images. And honestly, it can still be challenging to troubleshoot at times.

Async processes create a ton of timing issues that you just don't deal with in most of ServiceNow's overwhelmingly synchronous code base.

What do I wish I had done differently?

Flow Designer. I wish I had used Flow Designer. No, this isn't some shameless product shilling on my part. Don't get me wrong, I love a good shameless plug. This just isn't one of them.

Flow Designer operates async by default and it's really good at it. It supports executing as other users, which is probably a more secure option than running as the System user (although it can do that as well). It has traceability built in. Honestly, it's probably why Integration Hub exists and why so many savvy integration folks I know love it. Flow designer would consolidate many of the scripts into more focused Flows that would likely be better organized.

And truth be told, it may be worth going back and rebuilding some of these scripts on Flow Designer. Kristy would be proud.