Removing Emojis from emails - without converting to String first

[Carl Fransen1]

Hi Team,

We receive a number of emails in our instance with Emoi's - Out of the box ServiceNow doesn't handle these and the emails are cut-off from the place where the emoji is, which means we lose critical information.

I implemented the below Business rule, before/insert with the code below which works great.

(function executeRule(current, previous /*null when async*/) { 
 
// emoticons truncate emails, so remove them prior to insert 
gs.log ("Cleaning up Email to remove Unicode"); 
 
current.body_text = current.body_text.toString().replace(/^[\0\uD7FF\uE000-\uFFFF]|[\uD800-\uDBFF][\uDC00-\uDFFF]/g, ' - '); 
current.body = current.body_text.toString().replace(/^[\0\uD7FF\uE000-\uFFFF]|[\uD800-\uDBFF][\uDC00-\uDFFF]/g, ' - '); 
	
})(current, previous);

But (there's always a but!) as the email is now converted to String the email loses it's HTML and now looks like the below:

Without the rule it looks like this - much friendlier for our users:

Can anyone shed some light on how I could remove Emoji's without converting to a String first?

Appreciate any assistance!

Thanks Carl.

[Carl Fransen1]

Hi Warren,

Thanks for your reply - yes I also tested with Kingston and noted that although the email side works now, the system still doesn't translate that to the ticket that is created correctly, but at least the entire original email is available for reference, previously this was cut-off.

I also believe that ServiceNow should fix this for the entire system, will look to escalate and log a ticket with HI to see if we can gain some traction on this.  It would be great if you could also highlight this issue (and anyone else reading this) so we can have some numbers behind this request.

Cheers

Carl.

[Warren Pilkingt]

Hi Carl,

Thank you for your response.

If you do raise a HI ticket, please let me know the number and I'm more than happy to comment on that and back you up, as I agree, it needs to be fixed.

As far as I can see it, there are a number of ways that it could be resolved:

• Have a working script which keeps the HTML which strips out the emojis correctly (the two supplied had mixed results)
• Allow for the journal list fields (comments / work notes) to be UTF-8 friendly, so no scripts needed

Ideally rather than us have to add scripts or amend field definitions, there should be an inbuilt feature to strip emojis from emails when received that can be configured in the email options.

[Warren Pilkingt]

Good afternoon Carl,

Just for curiousity, I set up a developer instance of London and kept it out of the box apart from allowing incoming email, and set up a SN account for me and a valid email address so I could send email to that instance.

I did that, and sent a test email with five emojis.  You'll note that the incoming email was processed correctly when you show the HTML body:

When the email is processed into the comments and work notes, note that most of those emojis are replaced with ???? directly:

I can't see anything in the what's new articles about this.  I'm suspecting there's a default inbuilt rule or setting to substitute any non-compatible emojis with four question marks instead - not that I've found it yet.

If you're able to fire up a development instance of London for yourself, I'd be interested if you have the same.

Regards
Warren.

[Carl Fransen1]

Thanks for testing that in London, I'll also try this once I have some time as if it's fixed than we can ditch our code when we upgrade....