Data relevancy in Now Assist and VA

[Thulasidharan]

Hi All, 
     

I have a few questions regarding data relevancy in Now Assist and the Virtual Agent (VA):

1. How are Now Assist and Virtual Agent trained?
   What kind of data is used to train these systems?

2. What are the sources of training data?
   Do they get trained using content from ServiceNow documentation, the Developer Platform, or some other data sources?

3. How frequently are these models updated or retrained?

4. Can we train Now Assist and Virtual Agent using our own custom data?
   If yes, how can we do that effectively?

5. I created a custom skill to summarize a capability, but the generated output felt inaccurate.
   The summary produced seemed like it was generating new content rather than accurately reflecting the actual capability details
   This issue becomes more evident when summarizing manually created capabilities (i.e., those not previously available in any documentation).
   When I used Now Assist to summarize such a custom capability, it returned content that was completely inaccurate and misleading.

Anyone please help on this🙂
Thanks!!

[PaulSylo]

Hi @Thulasidharan 

Here is the understanding from my end,

1. How are Now Assist and Virtual Agent trained? -

Response: NOW LLM within the GenAI controller, train the Data . also, it has data set form hugging face ( starcoder2) used for text to code. - https://www.servicenow.com/company/media/press-room/starcoder.html
2. What kind of data is used to train these systems? 

Response : this source says - existing data within the ServiceNow platform, including incidents, changes, problems, knowledge base articles, data organized in tables, like records in the platform, with defined attributes and relationships, chat logs, email content. 

I took this below form Now assist documentation, suggested data for skills

1. What are the sources of training data?
   Do they get trained using content from ServiceNow documentation, the Developer Platform, or some other data sources?Response : Yes, - https://www.servicenow.com/docs/bundle/xanadu-application-development/page/script/now-assist-for-cod...

2. How frequently are these models updated or retrained?Response : usually, any LLM are retrained regularly ( often weekly) to  maintain accuracy and relevance to avoid Data Drift..

3. Can we train Now Assist and Virtual Agent using our own custom data?
   If yes, how can we do that effectively?

       Response : Yes, we can train the 

1. I created a custom skill to summarize a capability, but the generated output felt inaccurate.
   The summary produced seemed like it was generating new content rather than accurately reflecting the actual capability detail

        Response :  this is highly possible as you have manually have some updates and that was not relevant . try to have text based on existing documentation. 

there are more, but this is whatever i know and learnt!

[Thulasidharan]

The custom skill which I want to summarize the capability, those are already existed in the servicenow docs portal, So my guess is that, It will be trained with those data right??

But when I ask to give a summary about that, it was irrelevant. The manually created capabilities will not have the data in any of the platform, it's ok!! But for the existing one, it is giving irrelevant summary.

1. Can we train Now Assist and Virtual Agent using our own custom data?
   If yes, how can we do that effectively?

       Response : Yes, we can train the 
Reply: How can we train the Now Assist custom skills or any AI agents in servicenow?