Bulk data migration from Neo4j to Service Now CMDB

[ramprasad1910]

What i the best way to migrate bulk data (more than 10 million records) from Neo4j to Service Now CMDB?

[David A2]

Speaking as a CMDB admin who is still, 10 years later, dealing with junk records imported from an old system is to do it thoughtfully and piece by piece. As the old saying goes, garbage in - garbage out.

• Consciously decide what records/data are necessary to transfer over and provide organizational value
• Export the data into logical chunks based on their target CI class
• Review ServiceNow CI class manager  identification rules for each class and ensure source data has that info
• Build a transform map(s) - I would personally recommend one per primary class/table you're importing into. Ensure you are using the IRE on your transform map. 
• Import records and run transform - make sure you review the logs for records that didn't import properly

If you want to do an API and create CI's that way, you can, but in my opinion increases the risk of bad or misplaced data

Make sure your data is conforming to the CSDM and CI lifecycle modeling

[Ashish37]

In addition to what David has already said, create a Discovery source and create appropriate Reconciliation Rules for classes you want to bring data in.

I would suggest you to take a look at IntegrationHub ETL :

https://docs.servicenow.com/bundle/washingtondc-servicenow-platform/page/product/configuration-manag...

[ramprasad1910]

Thank you very much for your response. Should I go with Apache Kafka and push the data from Neo4j (external database) to Service Now? Or do we have any other efficient mechanisms to push large amounts of data? Additionally, I would also like to know if I can pass the data from Kafka to Service Now ETL. I am a newbie to Service Now and it would be great if I can get the best practices for handling this use case.