multiple solutions for same table

[Daniel Billing]

Good morning!
new to Predictive intelligence and was trying to figure out what PI solution to would used if you have 2 definitions with active solutions that have conditions that matches.
will it evaluate scoring in both and pick the highest?

[Musab Rasheed]

Hello,

In my opinion that's how it should work, if it is working like that then you are good.

[Brian Bakker]

@Daniel Billing 

It depends on the solution type... is it Classification, Similarity, Clustering or a Regression solution? The conditions (Filter) in a solution definition defines the training dataset to be sent to the ML Trainer for solution training. It is the Input Table and Fields on all solution types and in Classification/Regression solutions, also the Output Field that are important in defining what the solution does, and how it evaluates the prediction scores. If the solutions were exactly the same, it would produce the exact same prediction scores. Search the Community for "Tuning Predictive Intelligence Models" and read the articles to understand and tune these different solution types.

If you do have two Classifications solutions on the same Input Table and Output field with different Input fields and/or solution filter that produce different prediction scores, then you would need to create a client-side script or Business Rule that uses the ML APIs to trigger the predictions to check the prediction response from both solutions and see which solution returned the predicted value with the highest prediction score to populate the Output field on the Input table.

Hope this helps.

Regards,

Brian

[Daniel Billing]

Hi Brian,
It's a classification definition to set Assignment.
IRL we will update a single solution definition and tweak the input + conditions and submit for training.
in TEST instance i made a copy of the existing one, did my adjustments and submitted that for training to get the possibility to compare the results of the 2 solutions .
At this point I started to think about the order of running evaluation of multiple solutions and pick the one with highest confidence.
Would assume that there is a script somewhere that is running a query (or several) to find a solution record that returns a value... and would it be possible to decide an order?

[Brian Bakker]

@Daniel Billing 

Generally for Classification solutions, customers test many different variations of Input fields for their Classification solution with different advanced settings and use the Solution Statistics generated on the [ml__solution] record to determine which solution provides the best predictions and then deploy a single Classification solution in production.

To determine how good the trained Classification solution, you base it on the Solution Statistics generated for Estimated Precision, Estimated Coverage and Estimated Recall and there is also a dashboard with statistics for individual classes the solution training generated. With a training dataset, it trains on 80% of the data provided, and it generates the solution statistics by using the other 20% of the data to do predictions on the trained solution. All this is done during the solution training.

If you do want to run with multiple Classification solutions on the same Input table and Output field, then you get the prediction results from each Classification solution, as per the Get predictions using a solution version section and then script it to only use the prediction with the highest prediction score. Otherwise, it will just do the prediction on each solution, and it will use the predicted value from the prediction on the last solution that was invoked. Out-of-box, it will not do this evaluation for you and pick the prediction with the highest score, as we only expect one Classification solution to be trained/published per Input table/Output field.

Further material on tuning Classification solutions -

Tuning Predictive Intelligence Models (part 1)

Tuning Predictive Intelligence Models (part 2) 

Tuning Predictive Intelligence models: Precision, Recall and the Confusion Matrix