Do the thumbs up in prod affect LLM understanding
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-15-2025 04:32 AM
Hi all,
my team is currently testing the new patch 6 for NowAssist. Thumbs up/down seem to be training the LLM in Dev2. Something I'm wondering though is if the training thumbs up/down in prod will affect LLM training also meaning that if users are not clear on procedures or dislike a procedures may affect 'the truth' of the LLM.
potentially is that something that can be kept for dev2 for training purposes and have thumbs up/down show in prod but only as feedback that don't affect LLM training?
- 1,820 Views
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-15-2025 11:10 PM - last edited 3 weeks ago
Hi
Thumbs Up & Down are part of feedback loop in terms of LLM response.
====================================================================================
How does user feedback from thumbs-up or thumbs-down get used?
Thumbs-up/down feedback provides a high-level signal to help ServiceNow evaluate how generative AI models are performing across different skills and use cases. This input is used to identify general patterns or recurring issues that may inform future improvements.
However, this feedback:
Is not used for automated model retraining or fine-tuning
Does not lead to immediate, instance-specific changes
Does not guarantee resolution of any specific issue
Instead, it is reviewed by ServiceNow’s product and engineering teams to help prioritize model improvements over time—for example, identifying common points of confusion, formatting issues, or gaps in understanding that may affect multiple customers.
Feedback is only collected from customers who have opted into the Advanced AI & Data Product Terms data sharing program. When paired with usage logs and context, this feedback becomes more meaningful and helps our teams better understand and improve model behavior.
==========================================================================
How it goes in background its as below.
Have a look at This Table--> sys_gen_ai_log_metadata.
Data and Logs are too much encapsulated so not easy to decode . Have a look at table - SYS_GEN_AI_LOG_METADATA
Just tried Another Like - and 1 dislike in Chat Window and another entry appears for Both
Ane here are the entries for all the feedback.
2 like and 1 dislike
Just to Add here:-
Regards
RP
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
3 weeks ago
++ More Details