GenAi - performance testing

Lukasz_B · 3 weeks ago

I want o assess the performance of GenAI features (response time) (such as incident summarization and resolution notes generation) using the current Now Assist plugin.

However i do not see any straightforward tool that ServiceNow could offer.
Can you please help me find the solution to this needed functionality?

rpriyadarshy · 3 weeks ago

Have a look on this - NASK

The ServiceNow Prompt Performance tab within the Now Assist Skill Kit (NASK)is a tool for evaluating and optimizing the effectiveness of custom generative AI skills by providing metrics on correctness, faithfulness, and latency for a given prompt against specific data records. This tab allows you to collect data, run evaluation tests, and view results to understand how well your prompt is performing, and it can also initiate an automated prompt optimization process to improve the prompt's output.

Correctness: How accurately the prompt's output aligns with the source material.

Faithfulness: Whether the output generates unsupported conclusions or "hallucinations"

Latency: The time it takes for the AI to generate a response.

Sample Evaluation SS which is done using NASK for Your reference.

Regards

RP

View solution in original post

Lukasz_B · 3 weeks ago

Seems like what im looking for thanks!
Is installation of the plugin sn_skill_builder enough to run ?

rpriyadarshy · 3 weeks ago

HI Lukas

Please refer this FAQ on NASK it will have answer to all your queries.

How can I access the Now Assist Skill Kit?

To access NASK, you must adhere to the following criteria:

Have an active license for a Now Assist for [x] product.
You have updated the Now Assist for [x] plugins to the latest versions.
Have an instance that is on at least the Xanadu release

And as you said SKILL KIT Plugin

https://www.servicenow.com/community/now-assist-articles/now-assist-skill-kit-nask-faq/ta-p/3007953

Regards

RP