Built something you're proud of? Tell the story. A quick G2 review of App Engine or Build Agent helps other developers see what's possible on ServiceNow. Share your experience.

Incident creation through Chat test LLM appears strngly

User687957
Mega Guru

a week ago

Dear all,

I have setup a conversational assistant Twilio Agent in order to create an incident using a voice Agent.
The creation works great but wierd things happen

When testing this using chat input from Assistant Test button

fUsMjifHY8.png

This open an Agent panel where I can test my Agent either using voice or chat. For simplification I use chat mode first.

What is strange is as below :

  1. It ask me only 3 questions, while I am telling my LLM prompt to ask at least 5
  2. It did not ask me if the impact is affecting only me or all users but decide alone that it affect all users
  3. I ask the prompt to set the description as a summarization without Question/Answer format but It ignores it.

Here is below the prompt use :

chrome_byLxKdeHuP.png

chrome_PUaWhtyB7s.png

And here below the resulting created record

CJhUmPMFQw.png

As you can see from description it records it as Question/Answer format and the worst thing is that it has created 2 last question that it did not really ask me

 

Any idea how to improve and explain why it behaves like this ?

 

Regards 

0 Helpfuls
  • 232 Views
1 REPLY 1

rpriyadarshy
Tera Guru

a week ago

In My view LIST of Steps Section needs to be More Clear and in Simple Short Lines.

 

DID You Tried Changing the LLM model and Test with same Configuration?

 

It Seems that LLM is Hellucinating- Instruction Line 2 and 4 Is Ambiguous - WHat is enough Information.

 

Instruction line 3 needs more clear and Precise word for LLM.

 

Instruction Line 5 - Try Adding a Sample Format too Instead of writing again a long instruction for LLM

 

Regards

RP

0 Helpfuls