Possible workaround to extract data from ServiceNow

[pablor]

Hi all,

Could somebody advice how to face the extraction of huge amount of data from ServiceNow?

When trying to query tables with more than 200k records (including joins), our ODBC connector is not working as expected.

We have come with 4 different ideas how to get the data, but I will like to hear from you. Thanks in advance

• Possibility of getting database backup from SNOW?
• Snow Mirror tool
• Find a way how to use ODBC on tables with more than 200k records?
• How to raise limit of exported data from ServiceNow GUI?

[Robert Beeman]

I've pulled down more than 200k records using the ODBC, but if you are trying to do SQL joins then a local copy of database would really shine. I haven't used snow mirror, but I have been interested in it for my own company and it sounds like it fits your need. They have a 30 day free trial so why not give it a shot? It doesn't look like the trial supports MS SQL though... 😕

[pablor]

Hi Robert,

Trial version support MySQL and H2

If you go for full versions you will be able to use bellow list according to their documentation. I will write later once I'm done how was the experience.

Supported databases:

• Oracle (version 11g)
• MySQL (version 5.5 and newer, including community version - MariaDB)
• Microsoft SQL Server (2008 and newer)
• PostgreSQL (version 9.2 and newer)
• H2 (version 1.3.172. and newer)

Regards

[garrett_prochno]

Hi Pablo,

This is a common problem when moving large amounts of data, using ODBC/Web Services starts to break down in it's ability to scale due to the restrictions and overhead at that layer (i.e. semaphores, memory, row limits, etc). A better approach (and sometimes the only approach) is to push the data from within ServiceNow out. An enterprise solution that adopts this strategy is Perspectium Replicator: http://www.perspectium.com/

[tony_barratt]

Hi Pablo,

With Geneva there is the possibility of pushing data from the instance.

Export sets

Best Regards

Tony

[garrett_prochno]

The challenge with the Geneva Export Sets functionality is that it's using the ECC queue and a mid server to do so, performance may be better than SOAP but still has significant overhead and other issues. What happens when your endpoint is down? Your ECC queue just fills up? How does this scale with heavy discovery workloads?