Zip file contents validation in business rules

[wclim]

Hi all,

I am trying to develop a Business rule that does validation on a zip file contents, but I am not getting much progress, therefore trying to see if anyone have explored this or implemented something similar before.

Here is how the logic should go:
1) Whenever a zip file is uploaded into ServiceNow, ServiceNow will unzip the file to check through the file contents
2) For each file inside, if the extension is .txt or .csv and if the MIME-type matches the extension, proceed to the next file. If not, abort the attachment upload,

3) If any of the files are a zip file, apply the same zip file validation logic (recursion)

I am currently stuck in unzipping the zip file to iterate through the contents to do the extension and MIME validation.

Any idea?

Regards,

Wei

[yad_achyut]

Hello @wclim ,

You can follow the following Steps

Step 1: Create a Script include write the following code in it 

//Create function in script include
validateZipFile: function(attachmentSysId) {
        var gr = new GlideRecord('sys_attachment');
        if (!gr.get(attachmentSysId)) {
            gs.error("Attachment not found: " + attachmentSysId);
            return false;
        }

        // Get attachment name and MIME type
        var fileName = gr.getValue('file_name');
        var mimeType = gr.getValue('content_type');

        // Check if it's a zip file
        if (!fileName.endsWith('.zip') || mimeType !== 'application/zip') {
            gs.error("Attachment is not a valid ZIP file.");
            return false;
        }

        // Process the zip file
        return this.processZip(gr);
    },

    // Process zip file and validate its contents
    processZip: function(attachmentRecord) {
        var attachmentSysId = attachmentRecord.getUniqueValue();
        var zipFile = new GlideSysAttachment();
        var inputStream = zipFile.getContentStream(attachmentSysId);
        var zipStream = new Packages.java.util.zip.ZipInputStream(inputStream);

        var entry;
        while ((entry = zipStream.getNextEntry()) !== null) {
            var fileName = entry.getName();
            gs.info("Processing file: " + fileName);

            // Skip directories
            if (entry.isDirectory()) continue;

            // Recursively validate if the entry is a zip file
            if (fileName.endsWith('.zip')) {
                gs.info("Nested zip file found: " + fileName);
                var nestedSysId = this.storeNestedFile(entry, zipStream);
                if (!this.validateZipFile(nestedSysId)) {
                    gs.error("Validation failed for nested zip file: " + fileName);
                    return false;
                }
                continue;
            }

            // Validate file extension and MIME type
            if (!this.validateFile(fileName, zipStream)) {
                gs.error("File validation failed for: " + fileName);
                return false;
            }
        }

        gs.info("All files validated successfully.");
        return true;
    },

    // Validate individual files
    validateFile: function(fileName, zipStream) {
        var validExtensions = ['.txt', '.csv'];
        var isValidExtension = validExtensions.some(function(ext) {
            return fileName.endsWith(ext);
        });

        if (!isValidExtension) {
            gs.error("Invalid file extension: " + fileName);
            return false;
        }

        // Read the MIME type of the file
        var mimeType = this.getMimeType(fileName, zipStream);
        if (!mimeType) {
            gs.error("Unable to determine MIME type for: " + fileName);
            return false;
        }

        // Check if MIME type matches the extension
        if (
            (fileName.endsWith('.txt') && mimeType !== 'text/plain') ||
            (fileName.endsWith('.csv') && mimeType !== 'text/csv')
        ) {
            gs.error(
                "MIME type mismatch for file: " +
                    fileName +
                    " (Expected: text/plain or text/csv, Found: " +
                    mimeType +
                    ")"
            );
return false;
        }

        return true;
    },

    // Determine MIME type (custom implementation for MIME type validation)
    getMimeType: function(fileName, zipStream) {
        try {
            // Logic to detect MIME type, for simplicity return a mocked value
            // You can implement your own logic or use a library
            return fileName.endsWith('.txt') ? 'text/plain' : 'text/csv';
        } catch (e) {
            gs.error("Error detecting MIME type: " + e.message);
            return null;
        }
    },

    // Store nested zip files temporarily for validation
    storeNestedFile: function(zipEntry, zipStream) {
        var tempSysId;
        try {
            var tempAttachment = new GlideSysAttachment();
            tempSysId = tempAttachment.write('sys_attachment', zipEntry.getName(), null, zipStream);
        } catch (e) {
            gs.error("Failed to store nested zip file: " + e.message);
        }
        return tempSysId;
    },

Step 2: Create a Business rule on sys_attachments Table and make use of following code in it

var validator = new AttachmentValidator();
    var isValid = validator.validateZipFile(current.getUniqueValue());

    if (!isValid) {
        gs.addErrorMessage('The uploaded ZIP file failed validation.');
        current.setAbortAction(true); // Prevent the attachment from being saved
    }
else
{
gs.infoMessage("Upload Successful");
}

[wclim]

Hi @yad_achyut 

Thank you for your inputs, the usage of java.util.zip library worked to read out the zip file contents, however I believe your storeNestedFile function does not work.
The line where you used GlideSysAttachment.write have to be used against a GlideRecord in the first parameter, and using it against a 'sys_attachment' string will throw an error. And putting a ZipInputStream instead of a String that the function require should also give another error.

[wclim]

After trying out in my PDI, I am able to achieve this partially so far:

Using Business rules, this is what can be done:

• Check the contents of the main zip file (no nesting of zips - zip files within zip file)
• unable to save the files from the ZipEntry into sys_attachment (therefore only I cant check further than 1 level of zip nesting)
• Check file extensions without saving any of the file contents in sys_attachment
• Unable to validate the MIME of the file contents - can't find any external MIME library that will work in a Business rule/script include
• Check folders and nested folders in the zip file

If using flow designer's "Zip step" action, it should be possible to unzip more than 1 level of zip files, as I can save the zip contents into sys_attachment records for further processing, and the file extensions are checked against glide.attachment.extensions before saving.

One limitation I found is that unzipping the zip file using flow designer does not do any MIME-check when saving the file contents into sys_attachment, so files saved in this way will bypass SN's MIME-check, meaning MIME validation of zip file contents is still not possible so far from what I have tried.

Also, this Zip step will require IntegrationHub Professional, which I do not have.

If anyone is able to validate MIME-type of zip file contents and/or also save the contents in sys_attachment via business rules, I will appreciate greatly if you can share your experience in doing so.

Regards,

Wei

[wclim · Accepted solution]

After consulting with SN support and some of their internal DEV teams, we came to a conclusion that MIME-type validation of zip file contents is not possible at this time.

In my opinion, if the lack of MIME-type validation within a zip file is considered a security concern, the best way to address this is to just restrict zip file from being uploaded at all.
I have raised an idea for this feature and hope it can get incorporated into the product.
https://support.servicenow.com/ideas?id=view_idea&sysparm_idea_id=2f4c94ce472f1a9077748d01426d43d1&s...