Exporting Vulnerable item data to PowerBI

[Rashmi Bansal]

Vulnerable items are having lots and lots of data updated every hour and exporting it to PowerBI via export sets is failing because of the huge size of the file generated to be exported.

Please advise work arounds

[Ct111]

1. Divide the data into smaller batches by using filters or conditions in the export set.
For example, you could segment the data based on time (updated_on) or some other field.
Export smaller files more frequently to reduce the load on the system and Power BI.

Filter condtion could be

updated_on > gs.now().addHours(-1) // Export only the last hour's data

2. Instead of exporting all the data every hour, only export data that has changed.
Use the sys_updated_on or sys_mod_count fields to track changes.

3. Configure Export Sets to generate multiple smaller files instead of a single large file.
Use pagination (sysparm_offset and sysparm_limit) to divide the data.

sysparm_offset=0
sysparm_limit=10000

4. Reduce the export frequency to every few hours or once daily to manage file size.

[Rashmi Bansal]

Still this is not useful since the data is huge in every hour also