How is 1000 record limit on Remote Table defined?

[Foster Hardie]

The documentation stipulates that we should not add more than 1000 records to a remote table. There is also a hard limit, but it is not documented. It is not clear if the limit is from a sys_properties record or if it is hard-coded somewhere.

BTW, our use case is to show data from cmdb_ci, then call an API to embellish it. We will have other use cases when we don't call an API at all, but want to take advantage of the on-read manipulation available in Remote Tables. We can add our own throttle to make sure we're using the API reasonably, and providing adequate performance for the user and would like the option of showing more than 1000 records in certain circumstances.

[asifnoor]

Hi,

Are you sure there is a hardcode limit ? As far as i know, It is a recommendation because all the data that you retrieve from remote table resides in memory and hence it is asked to keep the limit to low. Never heard about hard limit. Also, check if there is a out of memory error after a certain number of records are fetched.

[Foster Hardie]

Here's some simple sample code. I put a limit in because we have magnitudes more ci data than 1000. Create a simple table with nothing but sys_id and name and try this out (change it to task if that's more appropriate for volume):

(function executeQuery(v_table, v_query) {
var ci = new GlideRecord('cmdb_ci');
ci.setLimit(1111);
ci.query();

while (ci.next()) {
var row = {
sys_id: ci.sys_id,
u_name: ci.name
};

v_table.addRow(row);
}
})(v_table, v_query);

[marclindsay]

Did anyone ever get an answer on this? I am expected 1,039 records returned and every time I view a list of records for the remote table I get exactly 1,000 records returned. 

There has to be a value in the system somewhere that is limiting the rows returned. 

[Foster Hardie]

Yes. I should have updated this here when I learned this.

Create a new sys_property record "glide.script.vtable.max_rows" and set a higher limit there.

HI (and common sense) cautions strongly to not get crazy with pulling rows from remote systems this way. I suspect it's a somewhat arbitrary limit, but they've said to stay under 10x (10,000) rows. Your mileage may vary based on concurrency load and the row width/size and API you're consuming.