Normalization Data Services duplicates

[Michael Lineba2]

I have a large number of duplicate Hardware Models after running Normalization Data Services.    For example 30,000 Latitude 5400 models.    Any suggestions on cleaning these up and what might be causing this?

[Rahul Priyadars]

Are you running - HAM - Hardware Normalization scheduled job ?

May i know corresponding to these 30K models which must be Created by Discovery is Normalized Values are mapped in your Hardware Asset?

What are the sources of the high volume model creation?

Regards

RP

[pratiksha5]

You can clean the data with a background script. uncheck cancel after 4 hrs. Do this in any other browser (the one which you are not working in or after business hrs and uncheck the rollback option.

var del = new GlideRecord('table name');

del.addEncodedQuery();

del.query();

while(del.next()){

    del.deleteMultiple();

}

Check the created on the records match it with any running schedule and turn them off until you find the cause. 

[Greg G3]

I know this is several months old, but check this knowledge article from ServiceNow.  The Data Normalization itself may be duplicating these models, and there is one checkbox you may need to remove to stop this duplication.  

We disabled this, and the duplicates stopped.  Any that were created, however, need to be cleaned up, but they seem to get arbitrarily linked to various assets/CIs.  In the article, it indicates you can contact ServiceNow support for assistance in the cleanup.

https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0692003