ServiceNow backend takes a giant leap making the platform more efficient -Introducing RaptorDB

[Harneet Sital]

Note : The content shared below is a summary of all the information available on the internet and insights shared at the Knowledge 2024 keynote. 

ServiceNow is making a significant upgrade to the foundation of its Now platform: the database. They're phasing out MariaDB and migrating all customers to a new database called RaptorDB.

This isn't just a minor tweak. RaptorDB is a fork of PostgreSQL, known for its advanced capabilities and superior performance. Developed by ServiceNow after they acquired Swarm64 (a PostgreSQL performance expert), RaptorDB boasts a significant speed boost – processing up to 93,257 transactions per minute compared to MariaDB's 35,121 (numbers referenced during Knowledge 2024 keynote)

                                                                                                           (Source Knowledge 2024)

The performance gains will vary based on usage intensity. ServiceNow anticipates the most significant benefits for large organizations already pushing the limits of MariaDB. Here's a breakdown of the expected improvements for various clients:

• Small customers: 26% performance increase
• Medium-sized organizations: 52% performance increase
• Large organizations: 75% performance increase
  (numbers referenced during Knowledge 2024 Keynote)
  

                                                                                                          (Source Knowledge 2024)

Beyond raw speed, RaptorDB offers greater flexibility. Unlike MariaDB (a MySQL fork), RaptorDB leverages PostgreSQL's architecture, allowing for more complex queries and the creation of custom functions for further optimization. While MariaDB has its strengths, the Now platform's growth and complexity necessitate a more robust database solution. 

With the Xanadu release and the backend database changes, the following areas will have a significant change in performance 

- IT Operations Management: 41% faster!

- IT Asset Management: 53% faster!

- Customer Service Management: 33% faster!

- Overall workspace load time increase across all workspaces: 25%

                                                                                                            (Source Knowledge 2024)

Migration Timeline

The transition is already underway, with some customers already on RaptorDB. ServiceNow expects to complete the migration for all customers in 2026.

Conclusion :

From personalizing online shopping experiences to optimizing smart power grids, ServiceNow's RaptorDB database will tackle all diverse business challenges. It will empower companies to analyze massive amounts of data, both historical and real-time, to identify patterns and prevent problems. This will further enhance the efficiency and scalability of the platform and will unlock true potential of AI, giving businesses the agility to succeed in today's AI-powered world.

[Daniel Oderbolz]

Hi @Harneet Sital 

Thanks for sharing this.

It is great that ServiceNow finally changes the RDBMS to one with more punch - PostgresSQL.

As an Ex Oracle DBA, I find it astonishing how many features ServiceNow has reimplemented on top of the DB and I really hope this migration is used to improve some of these early design decisions.

It is should be clear that if you implement something on a higher level of abstraction, it will be less efficient then implementing it at a lower level.

Here are some wishes concerning the use of the Database by ServiceNow I have going forward:

• ACLs (Added 2025-05-17): The current ACL system gets more and more complex, leading to real-word security issues (see e.g. Understanding ServiceNow’s May 2025 Query Range ACL Update: What You Need to Know). The major reason for this, in my oinion, is that the ACL system was added on top of the DB and not inside of the DB. Of course, changing that will lead to a lot of headaches and loss of functionality. But it will make the system much safer. At least giving a customer the option of changing the ACL system to "DB only" would be very interesting.
• Partitioning: So far, ServiceNow implemented a custom approach to partitioning large tables called "table rotation". That is why many features to not work (well) with tables like syslog. In the background, a query on these tables is execued as a large and typically slow "UNION ALL" query. If proper partitioning was used, the optimizer knows which partitions to use, which speeds up things. Also, I guess a lot of code can be removed from ServiceNow as a bonus.
  This is even security relevant as searches in the logs are much too slow today (sometimes prohibitively slow).
• No more issues with String vs. Integer and other data types: ServiceNow does not make full use of the data types of the Database by converting often to string. This causes all kinds of issues, among others that you often see the value 'undefined' (as a string) or the fact that you find no data in a list because you put "100" instead of "=100" in the list filter of an integer column.
  Of course, to really get the most of this, one would need to add a capability to code in a language like TypeScript (which compiles to JavScript).
• Recursive Queries: different from Maria DB, PostgreSQL supports recursive queries. This makes it simpler to deal with hierarchies like you have them in the CMDB. (Of course, one could even go a step further and implement the CMDB in a graph database like huge-graph for example - but that is a much bigger step).
• Regex Queries: in the past, it was possible to perform queries using regex. This would be very useful for example to search the log after a security incident. See also this idea: Re-Enable Regular Expressions (Regex) in Lists, Condition Builder and Encoded Queries. PostgreSQL supports this out of the box.
• Prepared Statements: Both to avoid sql injection and to speed up the database, it would make sense to enable the use of prepared statements in GlideRecord and GlideRecordSecure. This could for example replace functions like GlideRecord.addExtraField()
• Safer Index behaviour: in the past, if was possible to "loose" data if indexes were broken. With the new DB, the system should be made robust against such issues by improving the error handling and monitoring.
• Triggers VS. JavaScript Business Rules: Business rules run on a very high abtraction level (DB - Tomcat - Rhino). Even tough challenging from a security point of view, it would be good if simple Business Rules could for example be transpiled in the background to DB Triggers.
• DB Procedures: It would make sense to investigate if it would make sure to enable the developers to create procedures - for example to speed up business rules.
• Inheritance: PostgreSQL has a built-in inheritance feature. Here, it should at least be investigated if a migration makes sense.
• Geo DB Features: With PostGIS there is an excellent spatial extension for PostgreSQL. It could be very interesting to add such capabilities to ServiceNow.

I know this is a large list, but I hope that we as customers get the most out of the future database.

We discussed this here among the ServiceNow specialists, and I got interesting input: one colleague pointed out that these might have been deliberate design decisions to be independent of the RDBMS. The other colleague sent me this interesting link about Peregrine Four - this might explain why GlideRecord feels like programming COBOL: you can do all of this using a file if you really want to.

I get the point about the lock in (especially when it comes to using precedural languages on the database), but for many features like PARTITION, Prepared Statements or Regex, all relevant RDBMS Systems (even MariaDB) support it with minimal differences.

It will need courage to change these things, but in the end, it will save time, money and even benefit the environment.
Also this is an interesting ESG (Environmental, Social and Governance) Case as this will surely save the energy needed to provide ServiceNow which ultimately also saves costs.

Best
Daniel

[Aaron_Butell]

Will this only be available with the upgrade to Xanadu or will it be extended to earlier releases when it comes out?

[Harneet Sital]

As far as I have heard and from some statements made, the database upgrade is irrespective of the platform upgrades (family releases). Some customers have already received upgrades and based on comments made, all customers will be upgraded by the end of 2025. 

It's best to reach out to the customer account manager to understand the specific timeline for the client. 

Thank you, 
Harneet 

[Ned Chamberlain]

Please comment if you have an instance that has been through a migration to Raptor/Postgresql.

[Jason Siegrist]

When and how do we get it  ?

[Simon Hendery]

Hi @Jason Siegrist - This has been answered correctly by @Harneet Sital: Some customers have already received upgrades and based on comments made, all customers will be upgraded by the end of 2025. Reach out to your customer account manager for information on your specific upgrade timeline.

[Jason Siegrist]

Hey @Simon Hendery   

Where are my 225 instances in that rollout?

Where are the changes in HI portal showing me when this will happen? 

How can I accelerate one instance ahead of the others?  

I have been in the SN Echosphere since Calgary. 🙂  This is, without a doubt, one of the best things that has come to the platform.   Regretfully, beyond the announcement, I have been unable to get any answers about these questions. 

[Simon Hendery]

Hi @Jason Siegrist - When you say you've been unable to get answers, who have you asked?

[Matthew Fody]

Jason Siegrist,

Please contact your Sales Account and Support Account Managers for more information about the RaptorDB migration schedule and where you account is in that schedule. They should be able to provide you with any available news.

[Roberto P]

Am I to understand, that the data of my Washington instances, can be migrated from MariaDB to RaptorDB, now, and If so, is there a migration document between databases?