About how to keep logs

[Ereshkigal]

Hi.
I have a requirement in my project that I want to keep log records for the following periods.

・syslog：90days

・sysevent：180days

・sys_transaction：1year

However, I have heard that changing log rotation is officially deprecated.
If anyone has any knowledge about log storage, such as file output, please let me know.

[HIROSHI SATOH]

• File output: Writing log files directly to a local disk or network storage.
• Database: Storing log data in a database.
• Third-party log management tools: Using tools like Splunk or Elasticsearch to collect and analyze logs.
• ServiceNow Log Export Service (LES): This service allows you to transfer ServiceNow system logs to external systems.

To achieve your desired log retention periods, consider the following:

• Utilize ServiceNow Log Export Service (LES): Transfer necessary log data to an external system for long-term storage.
• Implement a third-party log management tool: Use tools like Splunk or Elasticsearch for flexible search and analysis of log data.
• Contact ServiceNow support: Consult ServiceNow support for specific guidance on achieving your desired log retention periods.
• Use custom scripts: Implement custom scripts to periodically delete log data. However, exercise caution as this could impact the ServiceNow system.

If you are good at scripting, you can also output the system log, compress it, and attach it to a record in a specific table to save it. You can probably find examples in the community.

[Robbie]

Hi @Ereshkigal,

Great question and something that is not often discussed or thought about.

High level, you're looking at baseline storage time of approx 56 days before it is recycled.

Referencing the last Support link I'm aware of (as below) and using the Transaction log (sys_transaction) as an example, this is calculated by the default rotation of 7 days x 8 rotations.  (7x8 = 56). Please note, this does differ per log/table.

Sadly there is no quick or easy way to configure or implement the changes you require. I'd also suggest raising a ticket with ServiceNow support with regards to your options available.

The system uses table rotation and table extension to archive older logs. By default, the system uses the following schedule to archive common logs: 

Common log archive schedule:
Table                                |Archive schedule |Rotations |Type
---------------------------------------------------------------------------
Event           [ecc_event]          |Every day        | 7        |Rotation
Queue           [ecc_queue]          |Every day        | 7        |Rotation
Event           [sysevent]           |Every day        | 7        |Rotation
Log             [syslog]             |Every week       | 8        |Rotation
Transaction Log [syslog_transaction] |Every week       | 8        |Rotation
Email           [sys_email]          |Every 30 days    | 8        |Extension

https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0721331

To help others (or for me to help you more directly), please mark this response correct by clicking on Accept as Solution and/or Kudos.


Thanks, Robbie

Also check Exploring Data Management: https://docs.servicenow.com/bundle/xanadu-platform-administration/page/administer/managing-data/conc...

Data Archiving: https://docs.servicenow.com/bundle/xanadu-platform-administration/page/administer/database-rotation/...

[Mark Roethof]

Hi there,

Where did you hear that changing log rotation is deprecated? Please share info on that.

Do wonder, why keeping syslog etc so long? Do be aware, that for larger customers that will be terrabytes (= licensing, be aware of that!) of log data en billions of records. Why would you want that? Is that even useable (answer: for larger customers no)?

What I usually see at customers, we are not extending the number of days, we are shorting the number of days compared to out-of-the-box. Because to be honest, 56 days of system logs? Who uses that? That's already too much.