How to Improve Answer Accuracy When Using Custom Skills in Now Assist Panel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2 hours ago
We are currently working on implementing a system that uses the Now Assist Panel to suggest responses to inquiries while referencing past inquiry records and knowledge base entries.
We are currently using the Now Assist Skill Kit to create a custom skill and implement the feature that suggests responses to inquiries; however, we have not achieved the expected accuracy and would like to seek your advice.
◆ Implementation Details
The configuration is as follows:
1. Create a tool (Retriever) that performs semantic searches on the knowledge base and custom tables
2. Execute the above tool within the custom skill
3. Pass the search results to the prompt as context and generate a response
◆ Observed Issue
Even when records relevant to the inquiry exist, the semantic search often retrieves records with low relevance; as a result, we are unable to generate responses that reference the intended records.
Consequently, response accuracy has not improved sufficiently.
◆Verifications and Countermeasures Already Implemented
To improve answer accuracy, we implemented the following measures:
・Set the LLM’s Temperature to a lower value
・Adjusted Retriever’s chunking and re-ranking settings
・Reviewed the semantic search index settings
・Modified the settings for indexed sources
References:
However, neither of these measures resulted in significant improvement.
◆Question
In a RAG configuration using the Now Assist Panel and Skill Kit, could you please provide any recommendations from the following perspectives to enable more accurate retrieval of past inquiry records and knowledge?
・Best practices for Retriever settings
・Methods for improving semantic search accuracy
・Recommended settings when searching custom tables
・Design considerations when using knowledge and custom tables together
・Logs and diagnostic methods to check to improve answer accuracy
Additionally, if there is any additional information we should verify to troubleshoot this issue, we would appreciate your guidance.
Thank you in advance.
