Upgraded to new version

Pat Surtan · ‎04-30-2022

We upgraded to Rome for Dev and Test. We are planning to upgrade Prod next. How can we test Prod when it is a live instance? We can't run ATF in Prod so how does everyone test their Production instance after a major upgrade?

Tony Chatfield1 · ‎05-02-2022

Hi, yes I would always test functionality like task and fulfillment in live production post upgrade. These test records should be identified where possible with something that identifies the upgrade, and you can cancel (or delete) the records once you have finished testing.

All staff should be aware of the upgrade and should not be operating as BAU during the upgrade change window, and so the maximum impact is an email or 2 that some users\teams need to ignore.

View solution in original post

Tony Chatfield1 · ‎04-30-2022

Hi, the normal process I would follow with 3 instances is (assumes a Prod change freeze is enforced during upgrade testing)

Clone Prod to Dev
Upgrade Dev
Review any upgrade errors, test Dev and resolve issues adding the fixes to pre and post upgrade update-sets as appropriate.
Clone Prod to Test
Deploy pre upgrade 'fix' update-set(s) from Dev
Upgrade Test instance
Review any upgrade errors.
Deploy post upgrade 'fix' update-set(s) from Dev
Test the 'Test' instance IE confirm functionality

Repeat the whole process from step 1-9, to be 100% sure your upgrade will run smoothly.

Then
Clone Prod to Dev - so you have a fresh pre-upgrade reference instance.

Deploy any 'pre' upgrade 'Fix" update-sets to Prod from Test.
Upgrade Prod
Deploy any 'post' 'Fix" update-sets to Prod from Test.

As the Upgrade has already be comprehensively tested in freshly cloned Dev and Test instances at least twice, you should not need to undertake any in depth testing in Prod and I believe that basic testing of core functionality (following your BAU processes) should be more than enough. This testing should be carried out within the Upgrade change window.

Many organizations follow this with an 'Early Life Support' model for a few weeks.
Where any issues identified as upgrade related, IE not reproduce-able in the pre upgrade Dev instance, are assigned directly to the team responsible for the upgrade for immediately investigation\resolution.

Pat Surtan · ‎05-02-2022

Hi Tony,

Are you saying that Dev and Test should be upgraded and tested twice so that we don't have to test in Prod?

Allen Andreas · ‎05-02-2022

Hi,

No, Tony is saying you would have already done this process 2-3+ times over due to cloning Prod to sub-prod and then doing the upgrade there (so dev = you've done this once, test = now you've done it twice, etc.).

Please mark reply as Helpful, if applicable. Thanks!

Please consider marking my reply as Helpful and/or Accept Solution, if applicable. Thanks!

Tony Chatfield1 · ‎05-02-2022

Hi, you should always test Production post upgrade, but having already comprehensively tested the upgrade in sub-production (and fixed any found issues) your Prod testing should be just to validate the upgrade, not to identify and fix issues.

Using this general process I am nominally 99.9X % sure my Production upgrades will not encounter unexpected issues; however as part of the upgrade process I would always test Prod for core functionality, integrations, major customization, any core security related visibility IE Client data visibility, HR visibility if you use it etc.
These are targeted tests carried, out inside the Prod upgrade change window, based specifically on the environments key functionality, which are normally fast and unobtrusive.

Regarding the number of cycles required in sub-production, this depends on any issues you find and your success in fixing them, 2 cycles is normally enough but this is not a set limit - do it until you are happy with the result.

Dev: Identify and fix any upgrade issues
Test: Test\validate any upgrade fixes made to Dev.
Prod: Test to confirm the upgrade was successful.