JSON API and REST API returning different results

garydanko · ‎07-21-2015

I have written a Ruby gem for ServiceNow that allows me to fully interact with ServiceNow. It currently uses the JSONv2 API. Everything works splendidly but it's slow and there is a record limit of 500. I recently discovered the REST API which is much faster and has a record limit of 10,000.

However, I am seeing inconsistencies in the results set between the two for the same exact call. I currently having debug output displaying the record count. I've tried to figure it out. It looks like the results get skewed as soon as I enter more configuration item names to the list. I am wondering if I am hitting the limit for GET requests or something.

JSONv2 looks like this:

./test.rb --ci "IDManager (IAM 2.0),UnboundID,HA TicketServer,Partner Authentication Gateway,Authentication (UniAuth),Web & Hosting Platform (WHP),IDFederation,Identity Data eXchange (IDX),IDProfile,AssetID,OAuth 2.0,Identity User eXperience (IUX),IDManager (IAM),Util Data eXchange (UDX),OAuth,Subscription Data eXchange (SDX),Identity Universal Service (IUS),IDAudit" --start 2015-04-01 --end 2015-08-01

[Debug] executing method: incident_find

[Debug] fetching https://foo.service-now.com/api/now/v1/table/incident?sysparm_display_value=true&sysparm_query=u_maj... (IAM 2.0),UnboundID,HA TicketServer,Partner Authentication Gateway,Authentication (UniAuth),Web & Hosting Platform (WHP),IDFederation,Identity Data eXchange (IDX),IDProfile,AssetID,OAuth 2.0,Identity User eXperience (IUX),IDManager (IAM),Util Data eXchange (UDX),OAuth,Subscription Data eXchange (SDX),Identity Universal Service (IUS),IDAudit

[Debug] JSON payload: null

4 records found.

REST looks like this:

./test.rb --ci "IDManager (IAM 2.0),UnboundID,HA TicketServer,Partner Authentication Gateway,Authentication (UniAuth),Web & Hosting Platform (WHP),IDFederation,Identity Data eXchange (IDX),IDProfile,AssetID,OAuth 2.0,Identity User eXperience (IUX),IDManager (IAM),Util Data eXchange (UDX),OAuth,Subscription Data eXchange (SDX),Identity Universal Service (IUS),IDAudit" --start 2015-04-01 --end 2015-08-01

[Debug] executing method: incident_find

[Debug] fetching https://foo.service-now.com/incident_list.do?JSONv2&displayvalue=true&sysparm_view=json_view

[Debug] JSON payload: {"sysparm_action":"getRecords","sysparm_query":"u_major_incident=true^cmdb_ci.nameINIDManager (IAM 2.0),UnboundID,HA TicketServer,Partner Authentication Gateway,Authentication (UniAuth),Web & Hosting Platform (WHP),IDFederation,Identity Data eXchange (IDX),IDProfile,AssetID,OAuth 2.0,Identity User eXperience (IUX),IDManager (IAM),Util Data eXchange (UDX),OAuth,Subscription Data eXchange (SDX),Identity Universal Service (IUS),IDAudit"}

36 records found.

garydanko · ‎07-23-2015

I solved it. One of my CIs has an ampersand in its name and URI encoding it does not encode the ampersand so it is tripping up the URL processing. Converting the ampersand to a percent code before making a call to the REST API solved the problem.

View solution in original post

garydanko · ‎07-21-2015

So you are seeing the same results? It is driving me crazy!

garydanko · ‎07-21-2015

I tried something new. I alphabetized the CI list like so. And the result set is what it would be without any date constraints. It looks as though the REST API is not accepting "^u_start_date>=2015-05-01^u_start_date<=2015-08-01" the way the JSONv2 API does.

./test.rb --start 2015-05-01 --end 2015-08-01 --ci "AssetID,Authentication (UniAuth),HA TicketServer,IDAudit,IDFederation,IDManager (IAM 2.0),IDManager (IAM),IDProfile,Identity Data eXchange (IDX),Identity Universal Service (IUS),Identity User eXperience (IUX),OAuth,OAuth 2.0,Partner Authentication Gateway,Subscription Data eXchange (SDX),UnboundID,Util Data eXchange (UDX),Web & Hosting Platform (WHP)"

[Debug] executing method: incident_find

[Debug] fetching https://foo.service-now.com/api/now/v1/table/incident?sysparm_display_value=true&sysparm_query=u_maj... (UniAuth),HA TicketServer,IDAudit,IDFederation,IDManager (IAM 2.0),IDManager (IAM),IDProfile,Identity Data eXchange (IDX),Identity Universal Service (IUS),Identity User eXperience (IUX),OAuth,OAuth 2.0,Partner Authentication Gateway,Subscription Data eXchange (SDX),UnboundID,Util Data eXchange (UDX),Web & Hosting Platform (WHP)^u_start_date>=2015-05-01^u_start_date<=2015-08-01

[Debug] JSON payload: null

36 records found.

garydanko · ‎07-21-2015

I have figured out what is triggering the inconsistency but it makes no sense. If I put the dates BEFORE the CI list in the query I get the correct results. If I put the dates AFTER the CI list they are not parsed. Is this a bug???

Dates after the CI list

./test.rb --start 2015-05-01 --end 2015-08-01 --ci "AssetID,Authentication (UniAuth),HA TicketServer,IDAudit,IDFederation,IDManager (IAM 2.0),IDManager (IAM),IDProfile,Identity Data eXchange (IDX),Identity Universal Service (IUS),Identity User eXperience (IUX),OAuth,OAuth 2.0,Partner Authentication Gateway,Subscription Data eXchange (SDX),UnboundID,Util Data eXchange (UDX),Web & Hosting Platform (WHP)"

[Debug] executing method: incident_find

[Debug] fetching https://foo.service-now.com/api/now/v1/table/incident?sysparm_display_value=true&sysparm_query=u_maj... (UniAuth),HA TicketServer,IDAudit,IDFederation,IDManager (IAM 2.0),IDManager (IAM),IDProfile,Identity Data eXchange (IDX),Identity Universal Service (IUS),Identity User eXperience (IUX),OAuth,OAuth 2.0,Partner Authentication Gateway,Subscription Data eXchange (SDX),UnboundID,Util Data eXchange (UDX),Web & Hosting Platform (WHP)^u_start_date>=2015-05-01^u_start_date<=2015-08-01

[Debug] JSON payload: null

Number Configuration Item Priority Opened At Duration Description

======================================================================================================================================================================

INC0010427 Identity Universal Service (IUS) Low 2014-06-20 03:55:06 5 min generic description

INC0010491 Identity Universal Service (IUS) Low 2014-06-20 14:06:38 0 min generic description

INC0010613 OAuth Low 2014-06-22 17:03:26 0 min generic description

INC0019750 Identity Universal Service (IUS) Moderate 2014-07-15 00:57:50 521 min generic description

INC0024082 Subscription Data eXchange (SDX) Low 2014-07-24 03:59:56 14 min generic description

INC0037751 IDFederation Moderate 2014-08-27 02:27:17 50 min generic description

INC0039709 IDManager (IAM) Low 2014-09-01 23:30:45 21 min generic description

INC0044265 IDManager (IAM 2.0) Low 2014-09-11 04:25:14 37 min generic description

INC0045781 Identity Universal Service (IUS) Low 2014-09-15 09:45:01 1971 min generic description

INC0058128 Identity Universal Service (IUS) Low 2014-10-13 01:44:07 280 min generic description

INC0058589 HA TicketServer Low 2014-10-13 20:48:48 94 min generic description

INC0062821 Subscription Data eXchange (SDX) Moderate 2014-10-22 11:37:32 16 min generic description

INC0076095 IDManager (IAM) Low 2014-11-17 23:39:26 21 min generic description

INC0083082 Identity Data eXchange (IDX) Moderate 2014-12-01 23:00:24 89 min generic description

INC0083172 OAuth Moderate 2014-12-02 03:25:31 546 min generic description

INC0088293 OAuth Moderate 2014-12-10 15:25:28 85 min generic description

INC0109597 Identity Universal Service (IUS) Low 2015-01-22 01:55:09 21 min generic description

INC0112390 OAuth Low 2015-01-27 10:19:22 125 min generic description

INC0123114 Identity Universal Service (IUS) Low 2015-02-17 20:30:05 4 min generic description

INC0133283 Identity Data eXchange (IDX) Moderate 2015-03-06 16:46:57 64 min generic description

INC0139764 IDManager (IAM) Critical 2015-03-18 13:47:24 71 min generic description

INC0144674 Identity Universal Service (IUS) Low 2015-03-26 13:17:39 26 min generic description

INC0145691 IDManager (IAM 2.0) Low 2015-03-29 02:28:42 245 min generic description

INC0145964 IDManager (IAM 2.0) Moderate 2015-03-30 02:20:46 6284 min generic description

INC0146114 Identity Universal Service (IUS) Moderate 2015-03-30 08:05:22 81 min generic description

INC0148295 Identity Universal Service (IUS) Low 2015-04-02 03:03:04 352 min generic description

INC0149026 OAuth Moderate 2015-04-03 09:28:12 70 min generic description

INC0166509 IDManager (IAM) Low 2015-05-11 04:02:51 1729 min generic description

INC0191565 OAuth 2.0 Moderate 2015-06-25 19:33:49 162 min generic description

INC0194372 OAuth High 2015-07-01 10:43:39 101 min generic description

INC0194510 OAuth High 2015-07-01 13:38:58 108 min generic description

INC0194563 IDManager (IAM) Critical 2015-07-01 14:50:30 18 min generic description

INC0196719 OAuth High 2015-07-07 09:07:35 50 min generic description

INC0196841 OAuth Moderate 2015-07-07 11:15:15 48 min generic description

INC0198694 Identity Universal Service (IUS) Low 2015-07-10 06:17:14 222 min generic description

INC0199792 IDManager (IAM) Moderate 2015-07-13 12:41:38 65 min generic description

Dates before the CI list

./test.rb --start 2015-05-01 --end 2015-08-01 --ci "AssetID,Authentication (UniAuth),HA TicketServer,IDAudit,IDFederation,IDManager (IAM 2.0),IDManager (IAM),IDProfile,Identity Data eXchange (IDX),Identity Universal Service (IUS),Identity User eXperience (IUX),OAuth,OAuth 2.0,Partner Authentication Gateway,Subscription Data eXchange (SDX),UnboundID,Util Data eXchange (UDX),Web & Hosting Platform (WHP)"

[Debug] executing method: incident_find

[Debug] fetching https://foo.service-now.com/api/now/v1/table/incident?sysparm_display_value=true&sysparm_query=u_maj...^u_start_date>=2015-05-01^u_start_date<=2015-08-01^cmdb_ci.nameINAssetID,Authentication (UniAuth),HA TicketServer,IDAudit,IDFederation,IDManager (IAM 2.0),IDManager (IAM),IDProfile,Identity Data eXchange (IDX),Identity Universal Service (IUS),Identity User eXperience (IUX),OAuth,OAuth 2.0,Partner Authentication Gateway,Subscription Data eXchange (SDX),UnboundID,Util Data eXchange (UDX),Web & Hosting Platform (WHP)