Remote Table missing data

[Casper6060]

So i am looking into remote tables for a customer.

An issue i am having is that in the payload i see all the data is returned correctly

I have set a record size of 20 in my rest message, so i will only get 20 records at a time.

If i just check those records with gs.info i see every record is always there as expected.

But when i use v_table.addRow(myObj) i see all 20 times it returns true, so all 20 rows should be added.

But when i check the table, only 17 rows are present, and whats even wierder is when i do sorting ORDERBYDESC the number of records in the table changes from 17 to 14.

Anyone know what this issue might be?

[JeremiP]

The changing count of records on the destination table when changing the ordering on SN side suggests that one of the records must have data that isn't accepted on the destination, aborting the process.

Can you see anything in the logs on the destination? Perhaps there's some field value validation done there, akin to Data policies in Servicenow?

[Casper6060 · Accepted solution]

Hi Jeremi

Thanks for the reply.

So i can say that i looked through everything for a long time and finally found the issue.

The issue was me not understanding exactly how remote tables tie into ServiceNow's table.

So my API called the endpoint and got for an example 140 results. But then the batchSize (returned records) was set to 20. Then when those 20 records where returned, then the search i actually did to return the data, like firstname = Casper then filtered the data once again and removed the records that did not fit the query. So my original 20 suddenly became 17 records. And because there where 140 records returned, then when ordering the other way, the last 20 records where returned and then once again filtered which then might return 14 out of 20 records.

But i did think it was invalid data that was then not being added so I checked every single insert using debugging and everything was added.

Thanks a lot for trying to help.