Limit number of records processed in transform map for a Data Load

[madhuv]

Hi ,

Do we have any ways to process only set of records (lets say 1000) using transform maps out of more number of records (lets say 5000) imported.

Regards

Madhu

[Ankur Bawiskar · Accepted solution]

@madhuv 

Create onBefore transform script and add this

(function runTransformScript(source, map, log, target /*undefined onStart*/ ) {

	// Add your code here

if(source.sys_import_row >= 1000){
error = true;
}

})(source, map, log, target);

Regards
Ankur

[madhuv]

Hi Ankur,

Thanks for your response.

I just have to reframe the question to meet the my requirement. Apologies, I did not catch this requirement before I posted the question.

The actual import is to Load User Data. What the team that shares the files says is there may be two rows may come for a user is some cases. In such case, they wish to consider the second row for a user as a single update for processing. 

So, the processing should happen only for 1000 users in a import .(there may be 1000+ rows for 1000 users).

Regards

Madhu

[Ankur Bawiskar]

Hi,

If there are multiple rows for same user record then the system will ignore the next rows if the target record is already updated because of previous rows.

Regards
Ankur

[Manal Aquil2]

Hi Ankur , 

And how can I increase the number of records that I can import via transform map. 

Ex--I have to 45k records to be imported but right transform map is importing 15k records at a time.

Please share your insights.

Thanks and Regards,

Manal Aquil

@Ankur Bawiskar