Massive number of duplicate Product Models

[Marcel H_]

We've been using Discovery in our environment for a while now, and get a good amount of data back, though it could use some additional tweaking.

One thing that I've noticed, and I believe that Discovery aids in the creation, is that we have a massive number of duplicate Product Models in our system. I am very confident that all but a few were generated automatically from Discovery, and we now have 513,330 product models in the product model table, most of which are duplicates.

As an example, I know for a fact that we have 2 Dymo LabelWriter Print Servers on our network, but we have 311 product models, all listed as "LabelWriter Print Server". Others that I've seen that are also obvious duplicates include 17,710 for "3825" (which appears to be referring to a Cisco 3825), or 81,874 for a "Catalyst 37xxStack".

How can I consolidate these duplicates into single product models that are associated with CIs and Assets, and ensure that these huge number of duplicates aren't created again?

[DaveHertel · Accepted solution]

Hi, have you looked in to using the field normalization plugin  To consolidate product models? Seems like a viable option...

Does this help?

[Marcel H_]

Thanks for the suggestion Dave, I hadn't thought about Normalization but it does make sense that it could be used to solve the issue.

I'm working on getting that configured for the Product Models table, and hopefully that will get rid of all the duplicates. I haven't worked with Normalization much, so it'll probably take some tinkering to get what I need if I'm reading the documentation correctly.

[DaveHertel]

Yep - understood, some tinkering probably needed to get the results you want, but it does work nicely to resolve all the numerous differences that are really the same thing.  HP is the poster-child for this problem with crazy derivates of mfg / model names that all mean the same thing! 🙂  Good luck...

[Marcel H_]

It looks like things are working well, just slow going as the database churns though all these records and does the normalization and coalesce jobs. Good news though is that after about a day of setting up rules and them running in the background, I'm down from over 530k records on the table to 293k so far.