Export huge amount of data

Kevin Paul
Mega Guru

Hi All,

 

I am trying to export data from cmdb_ci table, CI records that we use. The total records to export as CSV is 788000,

the CSV max limit is 300000. Can anyone please help me in exporting the records, how to separate them and export as batches? How to sort and export? I have already sort ascending by name of the CI and exported first 300000 records, please suggest me some tips.

 

Thanks!

1 ACCEPTED SOLUTION

Kevin Paul
Mega Guru

I used the method of sorting by sys_id, exporting a batch and then using the last sys_id from first batch to create a new batch and export it again. 

 

The following link helped.

https://davidmac.pro/posts/2021-07-02-sn-bulk-export/

View solution in original post

5 REPLIES 5

Lithesh
Kilo Sage

Hi @Kevin Paul ,
If that the case you can change this system property that has the count of maximum number of rows.

Navigate to sys_properties.list
search for "glide.platform_ml.api.csv_max_line".
open the record and change the value from 30000 to 788000.
and save the record.

Regards,
Lithesh

@Lithesh ,

 

Is that advisable? 300000 is the max limit I guess going above that will cause any issue?

It would take more time to export rather there won't be any problem.

@Kevin Paul ,
Note: Increasing the export limit may impact the performance of your instance, so it is important to carefully consider the appropriate limit based on your needs.