DocIntel Extraction Intermittent Failures - Now Assist Skill randomly failing after working perfect

[SindhujaM399336]

Hi Community,

Environment: Vendor Instance
ServiceNow Version: Zurich

Issue:
DocIntel extraction was working perfectly until
4 days ago. Now it exhibits intermittent behavior:
- Sometimes extracts and populates fields correctly
- Most of the time fails without clear error
- No configuration changes were made

Suspicion:
We believe the Now LLM model serving the
DocIntel skill may be unstable. We have two
warnings showing:
1. Non-compliant skills with fallback turned on
2. Upgrade supports only Now LLM Service -
Standard (previously using NowLLM LTS)

Specific Questions:
1. Has anyone experienced similar intermittent
DocIntel extraction failures recently?
2. Is there a known issue with NowLLM LTS to
Standard migration affecting DocIntel?
3. How do we identify which model is being hit
during each extraction attempt?
4. Is there a way to check Now LLM Service
health/stability logs?
5. Could the non-compliant model provider warning
be causing fallback to an unstable model?

Any help appreciated!

[DanielJ43996773]

Hey @SindhujaM399336 how are you? 

Yeah, this sounds pretty familiar.
I’ve seen something very similar over the past few days on Zurich as well.

The intermittent behavior you described, sometimes working, sometimes not, usually isn’t config-related, especially if nothing changed. It tends to point more to model routing or something unstable on the service side.

About the NowLLM part, the shift from LTS to Standard can definitely have side effects. I noticed that after the change, some skills started relying more on fallback, which made results less consistent.

That “non-compliant skills with fallback” warning is a bit of a red flag. In our case, it looked like requests were being routed to different models depending on availability, which would explain why DocIntel works one moment and fails the next.

On your questions:

Yep, I’ve seen similar DocIntel extraction issues recently.
Nothing officially documented (that I know of), but the LTS - Standard move does seem to impact behavior.
Visibility into which model is being used per request is still pretty limited. Logs might give you some hints, but nothing super explicit.
You can try checking system logs and any GenAI/Now Assist logs available, not perfect, but sometimes you can spot timeouts or weird patterns.

Honestly, yes, fallback could definitely be pushing requests to a less stable model.

If you can, I’d try to isolate or reduce fallback behavior just to see if things stabilize, that helped us narrow it down.

[SindhujaM399336]

Hey @DanielJ43996773 ,

Thank you so much for the detailed response — really appreciate you sharing your experience!

Good news on our end: DocIntel is working fine now. One of our team members switched the instance to a more stable model, which may seems to have resolved the intermittent failures.

We also looped in our manager, and they're planning to raise a case to get this formally documented and addressed on the service side.

Thanks again for pointing us in the right direction — your insights around fallback routing were spot on!

[DanielJ43996773]

My pleasure please consider mark my answer as solution!