Using Last Run Date time?

[urmilla]

Hi,

We are currently importing around 75k server records on a daily basis using two JDBC data sources. These data sources utilize one import set table and one transform map. The transforms take over 3 hours per data source to run, however we do run   other data sources   which bring similar quantity of data and the transforms re ni are

not that time consuming.

We have a date/time field being sent from our data source "Last Modified Date". We want to leverage this field for using the last run datetime functionality. If anyone has successfully implemented this functionality. Kindly guide us.

The wiki page gives us a gist but would appreciate a more real time example.

Best Regards,

Urmilla

[urmilla]

Essentially the main concern is when I am able to transform around 400 thousand records for a sperate data source in an hour. what coukd be the possible reason behind these server data sources consuming so much time. Could this be a source related issue? Because the records load fairly soon.

[Kalaiarasan Pus]

The number of business rules and other scripts configured and how they are configured will decide how much time it takes to transform the data.

[venkatiyer1]

Hi Urmila,

Check out my document on use last run date time if that helps you.

Use Last Run Datetime for a JDBC type data source