MFA broken after clone down

[Tim Grindlay]

Our Development and UAT environments require local login with MFA. We have a clone preserver setup for the user_multifactor_auth table and during our last upgrade to San Diego we cloned our DEV and UAT environments.

Post clone, we could log into the instances with our pre-clone DEV and UAT MFA codes from our authenticator apps fine. For an unrelated reason we needed to clone our UAT environment again (several update sets committed in error) and after this second clone (a few days after the initial clone) no-one was able to login using their MFA codes and we were effectively locked out of the instance.  

Luckily we had a clean up script that re-enables our email, and sends them to a shared email account so we were able to get a temporary MFA code (For accounts that had an email, because you don't get that option if your account doesn't have one!)

While troubleshooting we discovered that clicking the 'Receive a code via email' link, brings the old codes back to life. You don't have to use the code that was sent to the email. I also have a suspicion that an account that is locked out and then unlocked also brings them back, but I haven't thoroughly tested this as once the codes are working I have to find another user that had pre-clone MFA codes setup to test with.

I raised a case with the now Support but got nowhere. They suggested re-cloning, resetting the 'Enable multi-factor authentication' flag on the user profile or using the 'Receive a code via email' link, but these are all workarounds. Posting to see if anyone else has come across this.

[Charlotte Pakes]

Hi Luke, 

We are still waiting for ServiceNow to resolve a number of cloning issues we've been having recently or even confirming the results we found. But what worked for us regarding MFA is if you check in your Exclude Tables and Preserve Data records for duplicate entries for user_multifactor_auth and remove the duplicate so you only have 1 that seems to resolve the MFA issue. We'd had MFA setup and working for a few years successfully and it seems recently ServiceNow added OOTB records for the Exclude Table and Preserve Data records and having those plus ours we had added manually seemed to be causing the issue. Hope that helps.

Charlotte

[Luke43]

I've double checked our clone profile, we only have one exclude and preserver in the clone profile for user_multifactor_auth. (No duplicates). 

What's more interesting, I did a clone down but prior to doing so I purged the records for user_multifactor_auth on the target instance. After cloning was completed, I was able to sign in and enroll in MFA to the sub prod instance, however after signing out, I was unable to sign back in. When prompted for authenticator code and after entering, I was kicked back to the login screen again. I'm going to open a support case with ServiceNow and see if at least that can be fixed without having to roll back and clone down again. 

When we cloned down in Rome we didn't have these issues. This is our first clone down in San Diego though. 

[Luke43]

I've found a work around that works. Prior to clone down go to user_multifactor_auth table and set the MFA accounts that you need to sign in with to Validated = False. 

After doing this, we can use MFA with our local accounts. 

[Tim Grindlay]

Interesting - I'll have to experiment with this on our next clone down. Thanks for the update!

[NancyF]

The reason why this happens is that you've cloned the prod MFA onto the lower environment. To confirm, try to put in your prod MFA code and it should work.