Using Now Assist Skill Set and observations

[dheerajcheepati]

I have worked on Now Assist Skill Kit for past few days and thought of sharing few observations here -

      1. What is NASK ?

          You can refer to following FAQ documentation for details information Now Assist Skill Kit (NASK) FAQ - ServiceNow Community

     2. What Prompts did I try ?

         I tried varied prompts but would like to highlight two most interesting of all of them.

       a. AI Prompt to request for all active Service portals within the platform and explain how they are used.

          The response was satisfactory where I received good detail on active portals, their usage and and as well as additional parameters that the LLM probably thought is useful in the context.

      b. The second prompt was primarily "to identify slow performing reports that are impacting critical stakeholders and predict their impact on the business for the next month." The LLM came up with further questions on 

      -- The list of reports that need to be analyzed. This could be a list of report names, IDs, or any other unique identifier.

     --The definition of "slow performing".

     --The definition of "critical stakeholders".

  After providing the necessary inputs, the LLM provided response but was referring to Random report names like Report A/Report B etc. It appeared like the LLM hallucinated as I couldnt figure out which reports it was referring to.

 To address this issue, I went ahead and modified the Prompt to include sys_id of reports in output generated. This will hopefully make LLM understand the context better and provide the results I am looking for.

  The results were definitely better but not upto the mark still. I did see Random sys_id's returned where were not available in the instance.

  I will probably continue to finetune my prompts to hopefully get the most appropriate results.

3. Based on some of these prompts and responses, I do acknowledge that that complexity of prompt does play a significant role and so does the data that will/willnot be shared with NowLLM.