The CreatorCon Call for Content is officially open! Get started here.

A large number of tables were automatically created in the ServiceNow instance

pandah
Kilo Contributor

I found that in my instance, starting from a few years ago, a table called u_logtestxxx is automatically created.

More than 100000 tables have been created....

I don't know why this happened
The name of the table may even automatically increase
u_logtest01 until now u_logtest015199

I have reviewed all the scripts and did not find any automatic table creation processing.

 

At first, I extended the syslog table and created u_logtest01. Initially, data may have been inserted into this table, but later it became obsolete.

Later on, the u_logtestxxx table was continuously automatically created, and even the syslogxxxx table was created

 

 

I don't know how to troubleshoot this issue ..

 

 

 

 

2 REPLIES 2

M Iftikhar
Giga Sage

Hello @pandah,

This is a very critical issue that needs immediate investigation, as having 100,000+ tables can severely impact performance and stability.

The behavior you are seeing strongly suggests that a piece of automated code is running recursively or on a high-frequency schedule. The continuous creation of tables like u_logtestxxxx (and even base syslogxxxx tables, which is very odd) points to a problem with how your system is handling logging or data retention.

Here is a focused plan to diagnose and stop the issue:

 

1. Immediate Action: Stop the Process:

The priority is to stop the creation of new tables. Since you couldn't find a script, look for high-frequency or asynchronous processes:

  • Check Scheduled Jobs (All):

    • Navigate to System Definition Scheduled Jobs All.

    • Filter by Last Run Time (sort in descending order) or Run Once/Interval.

    • Look for jobs running every few seconds/minutes or any job that seems related to logging, data cleanup, or ETL/Integration. Disable any job that looks suspicious and has a high run frequency.

  • Review Flow Designer / IntegrationHub:

    • Check for any high-frequency Flows or Subflows that include an action like "Create Record" targeting a table name that could be dynamically generated.

 

2. Deep-Dive Troubleshooting (What's creating the tables?)

The key to finding the root cause is tracing the table creation event itself.

  • Audit the sys_db_object Table:

    • This table stores all table definitions. When a new table is created, a record is added here.

    • Go to sys_db_object.list in the filter navigator.

    • Look at the Created by and Created columns for one of the recently created u_logtest tables. This should give you the User or the System Process that executed the action.

    • If the user is System, the process is likely a Business Rule, Script Include, or Scheduled Job.

  • Analyze the syslog and syslog_transaction:

    • Since the creation is continuous, there must be a huge volume of log entries. Filter the logs around the exact time one of the new tables was created.

    • Search for keywords like createTable, db_object, GlideTableManager, or the exact table name prefix (u_logtest).

 

3. The Most Likely Root Cause:

The continuous creation of tables with numeric suffixes is the classic sign of a process attempting to:

  1. Insert data into an archived/obsolete table: Your initial setup where you extended syslog to create u_logtest01 is the critical clue. If a background process or an external integration is attempting to insert data into a table that no longer exists, a badly written error handler might be incorrectly attempting to recreate the missing table, often with an incremented name to avoid conflicts.

  2. Improper Archiving or Data Retention Policy: Check your Table Rotation or Archive Rules. If a table rotation rule is misconfigured or a script is incorrectly trying to rotate the original u_logtest01 table, it could be creating thousands of rotational targets.

 

4. Next Steps & Clarification Needed:

To help the community or ServiceNow Support assist you further, please clarify the following:

  • What is the exact content/code of the old u_logtest01 script? (If you can find it)

  • What version of ServiceNow are you running?

  • Do these 100,000+ tables contain any records? (Check by opening a newly created table like u_logtest015199.list)

Given the severity, I highly recommend opening an Incident with ServiceNow Support immediately. They have deep-level access to the database logs and internal diagnostics that can quickly identify the offending process.

Best of luck with your troubleshooting!
If my response helped, please mark it as the accepted solution so others can benefit as well.

Thanks & Regards,
Muhammad Iftikhar

If my response helped, please mark it as the accepted solution so others can benefit as well.

Hello @M Iftikhar ,Thank you for your answer
1.The tables are all empty and have no records
2.The creator and uploader of the table are both system
3.script:

   I have searched for the following script
   - Client Scripts

  - UI Scripts

  - Business rule

  - Script include

  - Schedule job

  - UI Actions

  - Scripted REST APIs

  - Script action

In the end, I only found relevant scripts in one script include. But the script was not called anywhere, and the content of the script only uses GlideRecord to query and insert record into u_logtest01

4.Scheduled Job
I did not find any execution history in the scheduled job dashboard on the date when the table was automatically created.

5.Some other information about the table

u_ logtest 01 extends the syslog table. The initially automatically created table (u_ log test 0001-0007) will extend syslog 0001-0007 (which appears to be a real system table). Subsequently, the newly created table will automatically create a blank table syslog0008-xxxx, and then extend this empty table to create u_log testXXXX

 

6.Although a long time has passed, in my impression, I have never created a table through script.

u_logtest01 seems to be just a simple table, the only difference being that it extends the syslog table.

There seems to be a behavior called Table Rotation, I don't know if it's related to him