Records missing from Table API results

[MikeD314]

I have been developing an application automate data exports from ServiceNow. Mostly everything works great but there is one issue that has been been bothering me for some time.

When querying the Table API often response pages do not contain the number of records expected. I believe this is due to ACL restrictions. The problem is that these hidden records are not reflected in the X-Total-Count HTTP header. Therefor when my application checks that all of the records exist in the local repository it finds a mismatch. Those records can't be accessed so it throws it into a loop of trying to fix the missing records until it gives up (using a simple max_retries count).

Is there any way to determine the number of records that have been hidden from a result page? I know the web interface will (at least sometimes) show the info as "X number of rows were removed by security constraints". But that information does not seem to be present in the raw JSON data or in the response HTTP headers.

My only idea for now is to keep a count of how many records are expected per page and then how many are missing. This will probably work but it seems overly hacky when ServiceNow knows the number of records it has hidden. This problem is also present when accessing the Attachments API. (I assume it will affect other web service calls as well but those are the two I've worked with so far.)

Is there any way to get the number of records hidden (overall or per-page) for a Table API request?

[MikeD314]

So basically I need to keep track of the number of missing records myself? I guess that's a solution.

I was hoping that there was some way to get an accurate record count directly from ServiceNow but it sounds like that isn't going to work.

[asifnoor]

While i know its a pain, but thats the only way as i see it.

Mark the comment as a correct answer and helpful if this answers your question.

[Chandu Telu]

Check the all read acl's on the table and check the user access(which your using as credentials)

if user may need some other roles to access the information. If your using admin access check the all read access where "admin override" should be checked. 

[MikeD314]

Where can I access the ACL definitions? (Is there a table that stores them?)

I'm trying to avoid making my application a huge security risk by forcing it to ignore security rules. I'd much rather work with them correctly. (There could be very good reasons to block some records from being replicated and ACLs are probably the correct tool for that in at least some cases.)