I haven't done a similar exercise but worked on domain separated instance for a good amount of years. Activating domain separation in an already live instance seems not to be a problem but as Tom mentioned you need assessment of data & code. The moment you activate domain separation in single tenant instance all the data & code will move to global domain but then it also means that data & code is visible to all domains.  You need to further assess what data like foundation data needs to stay common & what data need to be in domain for only A to have visibility to. Also same process to be followed for code e.g. a business rule implemented to achieve a particular functionality only meant for domain A needs to be in domain A & not in global. Similarly you need to have a migration plan for domain B with respect to their specific data & code. Domain visibility at group level can also be used to grant cross domain visibility as needed. This is going to be a massive exercise and like any change involves risk.

Wish you good luck & let me know if you have specific question