Predictive Intelligence-Classification Solution is getting Failed after reaching 100% or timeout

[skumarsingh4]

Hi Team,

I am working on a use case and created few classification solution for Category and Sub-Category of Incident with 3 months of Data. 3 months of Data consist of 800K rows. It worked perfectly fine and solutions has been created successfully. But When I use the same Data(with same filters) and try to create a solution for Another field of Incident i.e. "Assignment Group". It fails. I am attaching screenshot of the training progress. Most of the time training is failing with Time Out or it is getting cancelled after reaching 100% in 4th step and before uploading the trained solution. See the attached screenshot for reference. Do let me know if any additional information required from my end.

#Predictive

[Lener Pacania1]

Please open a ticket with support and ask them to dump the log into the case.  The training is happening on our AI infra so you won't have visibility into the logs from your instance.

[Brian Bakker]

@skumarsingh4 

Predictive Intelligence will only train 300K records newest records, as this is the current limit, even if the training data consists of 800K records. It took nearly 4 hours to remove duplicates from your training data, so you have a lot of "duplicate" data where the data in the Input fields are very similar or the same. By the time it removed all the duplicates, it may not have enough data to train the class. You may want to try using TF-IDF instead of Paragraph Vector as an advanced parameter. If the issue persists, raise a case with Support.

[skumarsingh4]

@Brian Bakker  Thank you for looking into this, we are working on a use where we need to train the model on few months of data, And we have already increased the training record size limit from 300K to 900K. As mentioned earlier. I am using same data filter and able to create the model successfully with(800k records)  for other output features of the incident, Like category or sub-category we are facing this issue only when we are training it for "Assignment Group" . It took 22hours 59minutes for Successful training of classification solution for other feature like category, sub-category with same 800k records. Attaching the screenshot of training progress for sub-category. As suggested by Lener have open a case for this issue.

[Brian Bakker]

@skumarsingh4 

Increasing the limit from 300k to 900k in the UI is not going to do anything, as we limit the size to 300k in the datacenter. Our documentation clearly states this in the Filter section -

Note: The recommended number of records for training a good solution is from 30,000 through 300,000. If you submit more than 300,000 records, the most recent 300,000 records are used to train the solution. Use only authentic records from the database.