What to do when Text after Emojis is Being Truncated

[Jisamma K]

On instances before Kingston, if an email contains special characters, like for example a smiley icon, all content after that character will be missing when stored in the system. This doesn't happen all of the time so it can catch someone off guard if mid sentence, your text gets truncated at the Emoji (😀🤫🤮).

Truncated emojis mid sentence

So why is your text being truncated after using an emoji? Well,  the reason for it is because the “String” field type in the platform does not officially support emojis. How to identify your text is being cut off after Emoji usage

If you or your customer tries to edit an email field or journal field and includes a sassy emoji (sometimes known as emoticon) they may find that the text following is cut off or truncated. 

How to determine this effects you:

• User types a sentence in email field.
• User inserts Emoji mid sentence.
• All text following emoji does not appear. 

What to do when your text after Emojis is being truncated

The “String (Full UTF-8)” field type does, and should be used if we expect to use emojis in that field. There is a slightly more overhead when using the String (Full UTF-8) type.

Our email form (sys_email) has support for Four-byte UTF8 Unicode characters on the subject, body, and body_text fields on Kingston releases and newer.

However, if an inbound action or processing is triggered, and the string contains Four-byte UTF8 Unicode characters, you need to ensure your target record fields support Four-byte UTF8 Unicode characters to avoid truncating data on those fields. The field type 'String (Full-UTF8)' is capable of handling the 4-byte emoji's character in all current releases. 

Note: Please make sure to change downstream field types everywhere a special characters is expected or need to be retained.

For more information on using Emojis in ServiceNow see:

• Four-byte UTF8 emoji characters cause emails text to be truncated when stored in the database
• • Unable to support emojis in Journal entries – Workaround is available in Kingston. Pre-K, Journal entries (aka comments/work notes) could not use emojis even with the String (Full UTF-8) field.
  • After Kingston, this works by changing specific fields to the String (Full UTF-8) type
• String fields are truncated when an emoji is inserted – work in progress. 

[tony_barratt]

Hi Jisamma,

Thanks for your post, it certainly clarifies the issue.

If I understand correctly with regards to downstream field types, if there was an emoji in, say the body of an email, then if the body was copied into another field, perhaps additional comments, then this field sys_journal_field would need to be of type String (Full UTF-8)?

Best Regards

Tony

[Jisamma K]

Yes Toni, the target fields need to be on type String (Full UTF-8) to support Emoji's.

Regards, Jisamma

[Fredrik v S]

Keep in mind that this stops the fields from working in Agent Workspace...

[Dub Myers]

Could you elaborate and provide a bit more detail?

[Dub Myers]

A minor note here is that while email is a common way for these characters to come into the system, this post is also applicable to any functionality that requires emoji character support, whether those characters arrived by email or by typing them in somewhere else.

And since, as of Kingston and later, the email table's fields storing subject, message (text and html versions) are updated to String (Full UTF8), the email table does retain the full text even if the downstream field was truncated.

[SN_Learn]

Hi @Jisamma K @Dub Myers 

I am using the Outlook Actionable Message and in the Survey Questions(type - Likert Scale) > 'Assessment Metric Definition' using the below (emoji+text)

The issue is that when end users are receiving it in outlook mails, these are getting truncated most of the times.

Could you please assist with this issue? Thanks