ServiceNow Database Growth & Record Deletions

[Kevin A3]

We are currently in the process of looking at our database size growth trajectory for our Live instance, and we need to reduce the size down considerably by looking at deletion of records, on top of the archiving rules against the main task tables (incident, problem, change).

When we go into ServiceNow Impact, we can see that the DBI Size (TB) increases gradually on a weekly basis, then dips, and repeats. This growth is taking place at a 0.01TB rate, and based on the trajectory, we will be hitting our limit in approximately 3/4 weeks.

Would anyone have any experience with this, and what you have done to reduce the database size ?

[Ken tang]

Been through this exact situation a couple of times. There's a few things you can do:

First thing to do before touching any archiving or deletion rules on task tables is check where your growth is actually coming from. In a lot of cases, the task tables aren't even the biggest offenders. The usual suspects are:

- sys_audit (almost always the largest table on any instance)
- sys_attachment and sys_attachment_doc (attachments are stored in 4k chunks, they add up fast)
- sys_email and syslog tables

If you haven't already, pull a storage breakdown report from Impact or have your Now Support team generate one. You need to know which tables are actually eating space before you start trying to reduce it.

After you've identified the culprits:

- Audit table cleanup is usually the biggest bang for your buck. Go to System Archiving > Archive Rules and look at whether you have aggressive enough rules on sys_audit. A lot of orgs have either no rules or rules set too far out (e.g. retain 5 years of audit data when you don't need it).

- Attachments are huge. Check if you have integrations or email notifications that are attaching files to records unnecessarily. There's also a PurgeOrphanAttachments scheduled job that cleans up attachment records with no parent, worth running if it hasn't been.

- Email logs (sys_email) are often overlooked. If you're not actively using email log history, you can trim retention down significantly.

On the archiving side for incident/problem/change, the native archiving is fine but don't expect it to be fast. It processes in batches and the DBI size won't drop immediately. You'll see the saw-tooth pattern in Impact flatten out over time.

The Data Management Console is worth checking out for ongoing visibility into which tables are growing fastest week over week. Makes it a lot easier to stay ahead of it going forward.

[Tanushree Maiti]

Hi @Kevin A3 ,

We are still working on it.

Archiving the data -does not mean it will reduce the DB size , still it will hold the space.

So suggested approch would be - work with your business/client and per document type check agreed retention policy .

That way you can start cleaning the legacy data, post backing up some place which is business approved.

[Jiri Nadvornik]

Adding one more finding in addition to the other suggestions here. According to our experience, the biggest contributor to a steady database footprint growth are attachments, or rather the sys_attachment_doc table. Since users tend to attach higher resolutions of screenshots and videos, optimizing this table usually gives the biggest price/value ratio for your effort.

After many similar iterations with our clients, we have developed a tool SnowMirror Drive which moves the attachment data from sys_attachment_doc table to an alternative cloud storage, such as Azure, AWS, or Google Cloud, while still keeping the sys_attachment records. This means the users still see the attachments in ServiceNow and nothing changes for them, only the data is dynamically loaded when needed.

This results in a ~90% reduction of the attachment database footprint and keeps it constant there. You can check it out on our website or contact us, I'd be happy to do a demo for you.

[Tanushree Maiti]

Hi @Kevin A3 

Please speak with your ServiceNow vendor about migrating to RaptorDB. This migration can increase your available database capacity by approximately 60–70%.