Bad Bugs - Canceling Import/Exports causing MID to hang

[amit_dhuleshia]

Affected Customers

Customers that cancel import/export in Geneva or Helsinki will be affected.

Problem Symptoms

• CancelProbe is not getting processed - stays in the ecc_queue output ready state.
• Nullpointer exceptions in the MID Server logs
  • 07/29/16 20:18:11 (708) ECCQueueMonitor.5 WARNING *** WARNING *** No probe registered to cancel for a given agent correlator 5037ad6713xxxxxxx128144b03b
  • 07/29/16 20:18:11 (708) ECCQueueMonitor.5 SEVERE *** ERROR *** java.lang.NullPointerException
• MID Server slow in processing items or not processing items at all

Problem

Due to a PRB introduced in Geneva, CancelProbe throws a nullpointer exception. As a result this item never gets processed - even worse, it keeps getting processed over and over again. The nullpointer exception keeps getting thrown and it consumes worker threads on the MID Server and causes the MID Server to slow processing and never processes the cancelprobe.

Workaround

Find and update the ECC Queue records that are causing the issue.For example, following the scenario in the Steps to Reproduce of cancelling an impor:

1. Navigate to ECC > Queue.
2. Set the following filters:
   • Agent is the affected MID Server (for example, mid.server.my-mid-server)
   • Topic is CancelProbe
   • Queue is 'output'
   • State is 'ready'
3. For each record, set the value of State to 'processed' and update the record.

Note: When testing this workaround, you do not have to restart the MID Server. However, if the issue persists after all ECC Queue records are updated, a restart might be necessary.

Fix

  The issue (PRB665835) has been fixed in Istanbul and backported to GP8 and HP4. Please find more information on this KB0597640