File size X MB exceeded the max size allowed - Lesson Learned

[Chris McDevitt]

I wanted to drop a quick note on what not to do.... and what to do...
😉
A customer's imports were failing, and it was getting the error:
"File size X MB exceeded the max size allowed...."
I found KB0564441 in Support.servicenow.com:
https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0564441

Don't use this KB! DO NOT change the max import file size!

"Why?" You may ask... Well, the problem is that each Data Source will bring in a file, which will then be loaded into -->memory<-- to process the data. The real problem occurs when each Data Source (running in parallel in its own thread) loads its massive file into memory. The Data Sources then run the Node out of memory.... and then bad things happen.

What you need to do is reduce the size of the incoming files (i.e., smaller page sets), which is different for each VR scanner integration, but here is a start:
https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0863634

How do I know this? You might be wondering... well, because I made this mistake.

Sigh....

[Bla_ena]

Hi Chris,

Thank you for posting your lesson learned. It saved my time and afford.

I also experienced "bad things happening" after increasing the max allowed file size, not knowing that it's just making things worse.

In my case, reducing the max_delta_days parameter(as per KB0863634) in Qualys configuration from the default 365 to as low as 3 days finally allowed the KB integration to import the last 2 year KBs. The issue with Qualys Knowledge Base integration is that the import file size depends on a number of KBs last modified in a given time period. If I use the default settings for importing the Qualys KBs with last_modified_date > 2020 the import file gets far too big. And fine-tuning the integration settings is quite time consuming.

So thanks again!

[Troy Riblett]

Just logged in to bump this up as helpful. Same issue happened with us. Increasing the max size turned this from an outage that would recover in 30 or so minutes, to an outage that took hours to recover and took down all our nodes.

Our fix was to lower truncation_limit in the Integration Instance record from 500 all the way down to 100, since that determines how many records get pulled in per "page." (Though, max_delta_days parameter like Bla_ena mentioned may help as well).

We've since crashed again after doing a plugin update, but fortunately that only happened in test and we were able to disable the integration. So... be careful updating to 12.8.4 and make sure to do a test run in a lower environment.