SMS Delivery Issue to AT&T Devices

[Chad Hall]

Anyone else having issues sending notifications from ServiceNow via email to AT&T SMS devices? For many years we've used the Email2SMS functionality that nearly every US provider supports to send text messages to our users via their SMS email address. For AT&T these take the form of 'phonenumber@txt.att.net' or 'phonenumber@mms.att.net'. Beginning Friday 1/18/19 this stopped working about 85% of the time. Emails sent to these same AT&T devices work fine when sent from other sources besides ServiceNow. When a ServiceNow email doesn't get delivered we get a bounce back reply from 'general@smartmessagingsuite.com' with this generic statement in the body:

"<>
@MESSAGE@=1 033 ERROR - the request is not authenticated

Email2SMS message could not be accepted as user acxiom@service-now.com cannot be authenticated. Please check your settings before trying again."

After some research it appears that smartmessagingsuite.com is an AT&T messaging service so I know its getting to AT&T. Its next to impossible to get anyone from AT&T Support who even knows that Email2SMS even exists, so nobody has been able to help me. I opened a HI ticket and while they understood the issue I'm not sure they're getting anywhere with AT&T on a solution as its now been 5 days with no resolution. ServiceNow was able to see that our messages to 'phonenumber@txt.att.net' started getting redirected to 'phonenumber@smartmessagingsuite.com' on 1/18/19 but I have no explanation from anyone why that's happening, what it truly means, or what we can do about it.

Since it only seems to affect our ServiceNow notifications I can only assume that its a ServiceNow-specific problem with AT&T. And if that assumption is correct then I suspect it may be impacting a lot of you all as well. We rely heavily on text messages for our high priority incidents so this is kind of critical for us. Please let me know if you've encountered this or if you have any ideas on how to resolve it. And before anyone suggests that we just use the Notify plugin for direct SMS messaging through Twilio, we don't own that and its not in the budget. 

Thanks! I appreciate the help!

Chad Hall

Acxiom

[michaelmonocchi]

Hi Chad,

We are experiencing the exact same issue with AT&T. This started last week sometime or maybe the week before. I too have opened a case with ServiceNow. Very frustrating. Thinking the solution may have been an inaccurate SPF record, we updated ours, but that did not help.

It's obvious AT&T made a change in the interest of security for their customers but it is causing problems. We use text messages as part of our password reset for users and it hasn't worked because AT&T keeps bouncing our text alerts.

I just reached out to SN again this morning and was told they are still looking into it. This has to be affecting many organizations. Thing is, if you just send a text with Outlook or from another phone to AT&T, it works so it is definitely some issue between SN and AT&T.

Mike

[Chris M3]

Yep, we're definitely experiencing the issue also, with some notifications being delayed as much as 24 hours.  It is an AT&T issue from what we understand.  AT&T is basically rate limiting the emails to text interface.  So since SN is sending 1000's of messages, it gets rate limited, where your organizations exchange interface might not be at that time.

[Chad Hall]

So it seems others have noticed the issue. I'm frankly surprised its not getting more attention. As much as companies rely on timely notifications there is real risk here for missing a ticket and breaching an SLA, which could lead to unhappy customers and potentially financial penalties. I think this is a much bigger deal than they realize.

Another annoying side effect of the auto-reply from AT&T is that each reply was opening a very generic and useless incident to our helpdesk team. If you encounter this you can create a new inbound email action to ignore them. You could also handle it using mailbox filters.

[michaelmonocchi]

I think it is now. I reached out to SN this morning regarding my case and was told they are looking into it more. It is a problem which can vary from customer to customer depending on your dependency of this service. For us, users cannot use our automated password reset service and some of our staff get alerts from SN for Incidents etc....which are now not working.

Yes, when the bounce occurs, an Incident does get created which is one more thing to deal with. We are working on customizing our rules to do as you say and ignore the incoming email. I think SN just needs to reach out to AT&T and figure out what is going on and someone needs to make a change to get this to work.