SnowMirror vs ODBC driver

Casey12 · ‎12-10-2018

My company is experiencing performance issues with the ServiceNow ODBC driver. As I understand it, this driver is not a real driver -- it is merely a wrapper around the out-of-the-box (OOTB) ServiceNow SOAP API, which is the cause of its poor throughput. It was recommended to me to look into SnowMirror to resolve our throughput problem. As I understand it, SnowMirror also uses the OOTB ServiceNow SOAP API. But, somehow, it has better performance than the ODBC driver (or so it's claimed). Is this true? If so, how can this be, given that they both use the OOTB ServiceNow SOAP API (and are, therefore, effectively the same thing)?

SanjivMeher · ‎12-10-2018

The advantage you have with SnowMirror

Your reporting will be faster. The reason it is faster, because it pulls data in advance.

It also pulls the reference fields display value, which is an advantage, which is not the same with ODBC Driver. In ODBC Driver, you will have to create database views to get the reference's field display values.

You can connect any reporting tool such as Tableau to generate report out of it.

Disadvantage

You have to pay for it. They have Standard and Enterprise package. I think it depends on the size of the data you choose standard or enterprise.

The reporting wont be real time. Because a scheduler runs every day and will update the snow mirror database on a daily basis. So you will have the data only till yesterday.

You will need another database server to store these data. So additional maintenance of a server.

Please mark this response as correct or helpful if it assisted you with your question.

Slawek_Radziewi · ‎12-10-2018

We used to use snowmirror but it is too expensive.

We switched to solution that we've added scheduled exports of needed tables which we send to local server.

These exports are imported to our local server using MIDServer and than used for reporting.

Ryan153 · ‎12-11-2018

We also had similar issues and tackled in two ways.

1. We use SSIS (with the ODBC connection) for the data extraction and were experiencing a lot of timeouts and jobs that ran for a very long time. The first thing we did was to break each table extract into manageable chunks based on the width of the table. For example if we knew we had 30000 metric_instance records we might configure the ETL to pull 6 chunks of 5000 records each. For the Incident table we pulled only 2000 at a time because the table has a lot more records. We were running daily extracts for about 90+ SN tables and this made a big difference in our ability to collect records.

2. We began to use Perspectium Data Integrations which was a game changer, and is how we are handling all SN data extracts now. With Perspectium we are able to move records into a SQL Server DB very quickly and then use that DB as the source for SSIS ETLS which loads into the DW. End to end we can get an entire day's worth of records for 90+ tables from SN into the DW in 45 mins. In some cases we are even using real time extracts for some specialized reporting. I would estimate this model cut our time in half, provided additional capabilities for data access and cut down our on-call issues drastically.

We had also done a POC with CrossFuze but they were unable to successfully move the SN data we needed into our environment. That was several years ago so I would hope they have things figured out by now.

Hope this helps

Casey12 · ‎12-11-2018

As far as I know, all of these third-party extract tools (e.g., Perspectium, SnowMirror) all use the ServiceNow SOAP API. But, so does the ODBC "driver". So, how is it that there are performance boosts with these third-party extract / replication tools relative to the ODBC "driver"? I understand that ETL with the ODBC "driver" can be tuned, but I'd like to know if there is any inherent performance boost with these third-party tools, and, if so, how they achieve those boosts (given that they're all SOAP API-based).