MID Server File Cleaner
Summarize
Summary of MID Server File Cleaner
The MID Server File Cleaner is an automated monitor thread that runs within the MID Server to manage and clean old files in specific folders under the MID Server’s install directory. Its primary purpose is to maintain the size and quantity of files, preventing potential performance degradation of the MID Server.
Show less
The cleaner deletes files and empty folders that have not been accessed for more than 30 days. It runs automatically at MID Server startup and every 24 hours thereafter. To trigger an immediate cleanup, customers can restart the MID Server service.
Folders and Files Managed
- \agent\import
- \agent\export – holds files generated by the Export Set feature
- \agent\work\monitors\ECCSender\outputerror – stores ECC input payloads that failed insertion (since San Diego release)
- \agent\work\monitors\ECCSender\outputoversize – stores payload copies exceeding the size limit defined by
mid.eccq.maxpayloadsize(default 20,000,000 bytes; since Eureka/Fuji releases) - \agent\logs – specifically Java Flight Recorder (JFR) dumps (from Utah release)
Logging and Monitoring
The MID Server agent log records the starting and stopping of the File Cleaner monitor. For detailed diagnostics, enabling debug logging with the parameter mid.log.level=debug shows folder scans before deletion. However, individual file or folder deletions are not logged.
Disabling the File Cleaner
Customers can disable the MID Server File Cleaner entirely by setting the configuration parameter mid.filecleaner.disabled=true. This can be done via the MID Server form’s Configuration Parameters related list or by directly editing the config.xml file. By default, the File Cleaner is enabled (false if parameter absent).
Practical Considerations for ServiceNow Customers
- Ensure critical files in the specified folders are moved or archived promptly, especially files in
\agent\export, to prevent unintended deletion by the File Cleaner. - If you rely on payload error or oversize files for troubleshooting, review the cleanup schedule and consider disabling the cleaner temporarily.
- Use MID Server restarts to manually trigger cleanup as needed.
- Adjust logging levels to debug when investigating File Cleaner activity.
- Use the disable parameter cautiously to avoid uncontrolled file growth that could impact MID Server performance.
A monitor thread runs in the MID Server to clean up old files, to keep the size and quantity of files within the install folder manageable, and to prevent performance issues with the MID Servers.
Files and empty folders within the following folders may be deleted by this MID File Cleaner. Files last accessed more than 30 days ago are deleted.
It runs on startup, and then every 24 hours after that. You can force the File Cleaner to run by restarting the MID Server service.
Folders/files cleaned:
- \agent\import
- \agent\export - Used by the Export Set feature.
- \agent\work\monitors\ECCSender\output_error - Used since San Diego for ECC input payloads that cannot be inserted for any reason (see PRB1521761/KB0995569).
- \agent\work\monitors\ECCSender\output_oversize - Used since Eureka/Fuji for saving copies of payloads that are not sent to the instance because they are larger than the limit set by mid.eccq.max_payload_size, which defaults to 20000000 bytes (see PRB598334)
- \agent\logs - Specifically JFR Dumps (Utah release)
Logging
The MID Server agent log shows this monitor starting and shutting down with the MID Server.
2022-10-21T13:08:58.330+0200 INFO (MIDServer) [Main:265] Initializing MID Server
2022-10-21T13:10:36.527+0200 INFO (StartupSequencer) [Monitors:307] Enabling monitor: FileCleanerMonitor
...
2022-10-24T09:29:35.193+0200 INFO (MIDServer) [MIDServerState:160] MID Server stopping
2022-10-24T09:29:38.591+0200 INFO (MIDServer) [Instance:928] interrupting thread FileCleanerMonitor.21600
To see exactly when it runs, enable debug with parameter
mid.log.level=debug.
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\import
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\export
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\work\monitors\ECCSender\output_error
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\work\monitors\ECCSender\output_oversize
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\logs
There is no logging for individual file/folder deletes.
Disabling the MID File Cleaner
Export Sets write files to the MID Server agent\export folders. The idea is then that a custom implementation on the customer side takes those files and moves them elsewhere, so that these files only need to remain in the MID Server application's install directory for a short time.
The MID File Cleaner can be disabled completely for all folders, by adding the MID Server Configuration Parameter mid.file_cleaner.disabled=true. When the parameter is not present, which is the out-of-box state, it defaults to false, and the file cleaner runs.
On the MID Server form, click New on the "Configuration Parameters" related list, select mid.file_cleaner.disabled, and enter the value "true".
This can also be manually added to the config.xml file: <parameter
name="mid.file_cleaner.disabled" value="true"/>. See MID Server parameters for more information.