MID Server File Cleaner
Summarize
Summary of MID Server File Cleaner
The MID Server File Cleaner is an automated monitor thread running within the MID Server to manage and clean up old files in specific MID Server directories. Its purpose is to maintain the install folder's size and file quantity at manageable levels, thereby preventing potential performance issues with the MID Server.
Show less
The cleaner deletes files and empty folders that have not been accessed for more than 30 days. It activates on MID Server startup and runs every 24 hours thereafter. Restarting the MID Server service forces the cleaner to run immediately.
Folders and Files Managed
- \agent\import
- \agent\export – used by the Export Set feature
- \agent\work\monitors\ECCSender\outputerror – stores ECC input payloads that failed insertion
- \agent\work\monitors\ECCSender\outputoversize – holds payload copies exceeding the size limit set by
mid.eccq.maxpayloadsize(default 20,000,000 bytes) - \agent\logs – specifically Java Flight Recorder (JFR) Dumps from the Utah release
Logging and Monitoring
The MID Server agent log records the File Cleaner’s start and stop activities alongside MID Server lifecycle events. To view detailed run times and directory scans, enable debug logging by setting the parameter mid.log.level=debug. However, individual file or folder deletions are not logged.
Disabling the File Cleaner
You can completely disable the MID Server File Cleaner by setting the configuration parameter mid.filecleaner.disabled=true. This can be done via:
- The MID Server form’s “Configuration Parameters” related list (add a new parameter named
mid.filecleaner.disabledwith valuetrue). - Manually editing the
config.xmlfile to include<parameter name="mid.filecleaner.disabled" value="true"/>.
By default, without this parameter, the File Cleaner is enabled and runs automatically.
Practical Considerations for ServiceNow Customers
- Ensure any files in the
\agent\exportfolder are moved promptly by your custom processes to avoid unintended deletion by the File Cleaner. - Monitor MID Server logs at debug level to verify File Cleaner operations if troubleshooting file retention issues.
- Consider disabling the File Cleaner only if you have alternative file management practices in place to avoid MID Server performance degradation.
A monitor thread runs in the MID Server to clean up old files, to keep the size and quantity of files within the install folder manageable, and to prevent performance issues with the MID Servers.
Files and empty folders within the following folders may be deleted by this MID File Cleaner. Files last accessed more than 30 days ago are deleted.
It runs on startup, and then every 24 hours after that. You can force the File Cleaner to run by restarting the MID Server service.
Folders/files cleaned:
- \agent\import
- \agent\export - Used by the Export Set feature.
- \agent\work\monitors\ECCSender\output_error - Used since San Diego for ECC input payloads that cannot be inserted for any reason (see PRB1521761/KB0995569).
- \agent\work\monitors\ECCSender\output_oversize - Used since Eureka/Fuji for saving copies of payloads that are not sent to the instance because they are larger than the limit set by mid.eccq.max_payload_size, which defaults to 20000000 bytes (see PRB598334)
- \agent\logs - Specifically JFR Dumps (Utah release)
Logging
The MID Server agent log shows this monitor starting and shutting down with the MID Server.
2022-10-21T13:08:58.330+0200 INFO (MIDServer) [Main:265] Initializing MID Server
2022-10-21T13:10:36.527+0200 INFO (StartupSequencer) [Monitors:307] Enabling monitor: FileCleanerMonitor
...
2022-10-24T09:29:35.193+0200 INFO (MIDServer) [MIDServerState:160] MID Server stopping
2022-10-24T09:29:38.591+0200 INFO (MIDServer) [Instance:928] interrupting thread FileCleanerMonitor.21600
To see exactly when it runs, enable debug with parameter
mid.log.level=debug.
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\import
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\export
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\work\monitors\ECCSender\output_error
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\work\monitors\ECCSender\output_oversize
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\logs
There is no logging for individual file/folder deletes.
Disabling the MID File Cleaner
Export Sets write files to the MID Server agent\export folders. The idea is then that a custom implementation on the customer side takes those files and moves them elsewhere, so that these files only need to remain in the MID Server application's install directory for a short time.
The MID File Cleaner can be disabled completely for all folders, by adding the MID Server Configuration Parameter mid.file_cleaner.disabled=true. When the parameter is not present, which is the out-of-box state, it defaults to false, and the file cleaner runs.
On the MID Server form, click New on the "Configuration Parameters" related list, select mid.file_cleaner.disabled, and enter the value "true".
This can also be manually added to the config.xml file: <parameter
name="mid.file_cleaner.disabled" value="true"/>. See MID Server parameters for more information.