Horizontal discovery sensor "Payload was in an unexpected format" error

[Tony Copeland]

Good Afternoon,

I have a pattern that is failing intermittently while performing a load balancer discovery.  It works on some load balancer pairs, but fails on others with an error in the Horizontal Discovery Sensor "Payload was in an unexpected format, the payload will not be processed" around line 45 (var ci_data = JSON.parse(result.output); .)   The only difference appears to be the quantity of the pool members (~800+) returned to the platform as it works if I disable the pool members steps.  

Does anyone have any insight on how to capture this payload without modifying the sensor?  Since it happens after the discovery pattern completes, it is not in the discovery log and I don't want to break the sensor (javascript isn't my best language by far).  Does anyone know JSON.parse has any limitations?  I did see a 10MB limit mentioned in the integration guide, but I wasn't sure if that applied to discoveries.  I requested (from our internal team) and increased quite a few mid server properties and they haven't helped.  I'm just not quite sure where to take this issue, as our last Hi case with a custom/in-house developed pattern was challenging.  

Thanks,

-Tony

[MarryJen]

Hi Tony,

This can cause issues with `JSON.parse(result.output)` since it might exceed the payload limits. The 10MB limit mentioned in the guide likely applies here, but even smaller payloads can cause problems depending on the data structure.

To avoid modifying the sensor, you could try increasing the MID Server memory limits (you mentioned you did this, but double-checking wouldn't hurt). Another approach is to break the discovery into smaller chunks, discovering fewer pool members at once. This could reduce the size of the payload being processed.

You could also log the raw payload before it hits `JSON.parse()` to check what’s actually being returned. If you're able to capture the response before the failure, it might provide more clarity on what’s going wrong.

Finally, if these steps don't help, I'd recommend reaching out to ServiceNow support, especially since this issue seems tied to large payloads, and they might have deeper insights.

Hope this helps!