Problems when importing from CSV file or XLfile

[peterdurup]

I try to import data from a file formatted as either a CSV file or an XL spreadsheet.

Either way I end up in an error:   Connection reset by peer.

This happens if I try to load more than 4000 rows. I have tried the same with app. 700 rows and then it works fine.

My problem is that I need to load more than 3 million rows so chunking it up is not a way to handle it.

Trying to load a CSV it fails because there are now fields in Fuji where I can enter the delimiter code and header row. (I guess)

Does anybody know what is causing this.

System is running on Fuji

[arnabbose]

Try searching if there is any limit set in the system definition of your instance. It could also be a connection (network speed) issue with pulling large data.

[peterdurup · Accepted solution]

It seemed that it was a limit that prevented me from importing,

I added a new system property: com.glide.processors.XMLProcessors.record_count and set it to 400000, then I tried to upload from my spreadsheet again and succeded in loading 20.000 records.

Thank you

[LearnerSubho]

Hello Peter,

Go to System Properties->Import Export.

Change the settings for import or export row limit as per your requirement.

Thanks and Regards,

Subhankar

[peterdurup]

I tried that, and removed the new property. There are not much to change when talking about import:

I changed Error limit for CSV format to 1000 and set ignore non-parsable lines to true.

On EXCEL format set the memory percentage limit to 80

That did not help, the error came again.

Changing the Export properties for CSV format to 400000 didn't help either.

But thanks for you suggestion.