MID Server File Cleaner
Summarize
Summary of MID Server File Cleaner
The MID Server File Cleaner is a monitor thread running within the MID Server that automatically deletes old files and empty folders to maintain manageable file size and quantity in the MID Server install folder. This cleanup helps prevent performance degradation of the MID Server.
Show less
The File Cleaner deletes files that were last accessed more than 30 days ago from specific folders related to MID Server operations. It runs automatically at MID Server startup and subsequently every 24 hours. To trigger an immediate cleanup, restarting the MID Server service is sufficient.
Folders and Files Managed
- \agent\import and \agent\export – used by the Export Set feature.
- \agent\work\monitors\ECCSender\outputerror – stores ECC input payloads that failed insertion since San Diego release.
- \agent\work\monitors\ECCSender\outputoversize – holds payloads not sent to the instance due to size limits (default max 20,000,000 bytes) since Eureka/Fuji releases.
- \agent\logs – specifically Java Flight Recorder (JFR) dump files from the Utah release.
Logging and Monitoring
The MID Server agent log records when the File Cleaner monitor starts and stops with the MID Server. By enabling debug logging with the parameter mid.log.level=debug, you can see detailed timestamps of when directory scans occur. However, individual file or folder deletions are not logged.
Disabling the MID Server File Cleaner
The File Cleaner can be fully disabled by setting the MID Server configuration parameter mid.filecleaner.disabled=true. This can be done either through the MID Server form’s Configuration Parameters related list or by manually adding the parameter in the config.xml file. If the parameter is absent, the File Cleaner runs by default.
Practical Considerations for ServiceNow Customers
- Because Export Sets write files to the
\agent\exportfolder, customers should implement custom processes to move these files elsewhere promptly to avoid accidental deletion. - Regular automatic cleanup helps maintain MID Server performance by preventing excessive disk usage.
- Use the debug logging option for troubleshooting the File Cleaner’s operation timing.
- Disabling the File Cleaner is possible if you need to retain files longer but consider the impact on MID Server storage and performance.
A monitor thread runs in the MID Server to clean up old files, to keep the size and quantity of files within the install folder manageable, and to prevent performance issues with the MID Servers.
Files and empty folders within the following folders may be deleted by this MID File Cleaner. Files last accessed more than 30 days ago are deleted.
It runs on startup, and then every 24 hours after that. You can force the File Cleaner to run by restarting the MID Server service.
Folders/files cleaned:
- \agent\import
- \agent\export - Used by the Export Set feature.
- \agent\work\monitors\ECCSender\output_error - Used since San Diego for ECC input payloads that cannot be inserted for any reason (see PRB1521761/KB0995569).
- \agent\work\monitors\ECCSender\output_oversize - Used since Eureka/Fuji for saving copies of payloads that are not sent to the instance because they are larger than the limit set by mid.eccq.max_payload_size, which defaults to 20000000 bytes (see PRB598334)
- \agent\logs - Specifically JFR Dumps (Utah release)
Logging
The MID Server agent log shows this monitor starting and shutting down with the MID Server.
2022-10-21T13:08:58.330+0200 INFO (MIDServer) [Main:265] Initializing MID Server
2022-10-21T13:10:36.527+0200 INFO (StartupSequencer) [Monitors:307] Enabling monitor: FileCleanerMonitor
...
2022-10-24T09:29:35.193+0200 INFO (MIDServer) [MIDServerState:160] MID Server stopping
2022-10-24T09:29:38.591+0200 INFO (MIDServer) [Instance:928] interrupting thread FileCleanerMonitor.21600
To see exactly when it runs, enable debug with parameter
mid.log.level=debug.
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\import
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\export
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\work\monitors\ECCSender\output_error
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\work\monitors\ECCSender\output_oversize
2022-11-17T02:49:13.201-0800 DEBUG (FileCleanerMonitor.21600) [AFileCleaner:103] Scanning directory to delete files: C:\...\agent\logs
There is no logging for individual file/folder deletes.
Disabling the MID File Cleaner
Export Sets write files to the MID Server agent\export folders. The idea is then that a custom implementation on the customer side takes those files and moves them elsewhere, so that these files only need to remain in the MID Server application's install directory for a short time.
The MID File Cleaner can be disabled completely for all folders, by adding the MID Server Configuration Parameter mid.file_cleaner.disabled=true. When the parameter is not present, which is the out-of-box state, it defaults to false, and the file cleaner runs.
On the MID Server form, click New on the "Configuration Parameters" related list, select mid.file_cleaner.disabled, and enter the value "true".
This can also be manually added to the config.xml file: <parameter
name="mid.file_cleaner.disabled" value="true"/>. See MID Server parameters for more information.