GenAi - performance testing

[Lukasz_B]

I want o assess the performance of GenAI features (response time)  (such as incident summarization and resolution notes generation) using the current Now Assist plugin.

However i do not see any straightforward tool that ServiceNow could offer.
Can you please help me find the solution to this needed functionality?

[Simon Hendery]

Hi @Lukasz_B 

There are plenty of resources we can point you towards if you're looking to performance-test Now Assist in a sub-production instance.

But if you only have access to a PDI, unfortunately Now Assist functionality isn't available on PDIs.

[Lukasz_B]

Hi Simon
Im strictly interested in sub-prod and prod instance. Sorry for not providing enough details in the first place. 

[Mahathi]

Hi @Lukasz_B ,

The only alternate way is to register for a Now assist course in the ServiceNow University and try testing those features there.

If my answer helped in anyway, Please mark it as helpful and accept as solution.

Regards,
Mahathi

[rpriyadarshy · Accepted solution]

Have a look on this - NASK

The ServiceNow Prompt Performance tab within the Now Assist Skill Kit (NASK)is a tool for evaluating and optimizing the effectiveness of custom generative AI skills by providing metrics on correctness, faithfulness, and latency for a given prompt against specific data records. This tab allows you to collect data, run evaluation tests, and view results to understand how well your prompt is performing, and it can also initiate an automated prompt optimization process to improve the prompt's output. 

Correctness: How accurately the prompt's output aligns with the source material.

Faithfulness: Whether the output generates unsupported conclusions or "hallucinations"

Latency: The time it takes for the AI to generate a response.

Sample Evaluation SS which is done using NASK for Your reference.

Regards

RP