Your AI Has Never Seen Your Instance

[Chris Pearson]

If you ask a general-purpose AI to write a business rule against one of your ServiceNow tables, it will do it. Confidently. With a field name it invented. I've seen this first-hand many times.

That’s the failure I see more than any other when people start building on SN with AI. The model has read a lot of public ServiceNow documentation, or so you may think. It has never read YOUR data model. It doesn’t know about your custom tables, the columns your scoped app added, or that one weird field sitting on incident that nobody can remember why it was ever created or what it's used for. So it fills the gap with something plausible, and plausible is the worst kind of wrong because it looks great when you read that code.

Think about hiring a brilliant contractor and skipping the blueprints. They’ll build you something, and it might be beautiful. A gorgeous deck...on the side of your house that doesn't have a door.

The fix isn’t a smarter model. It’s context. At Naitiv we put real effort into three kinds of context before we trusted an agent to write much of anything:

First, the platform’s data model. The agent needs a way to check the actual tables and columns before it writes code against them. Not from memory. From a reference it can look up. Don't get me started on AI session memory (maybe that should be a separate post).

Second, your data model. Platform tables are only half the picture. Your custom tables, your scoped apps, the columns somebody added three releases ago: the agent has to see those too, or it will invent them.

Third, docs that match the release. Code written for an old SN release is a quiet way to get bad code. Whatever documentation the agent leans on should match the version you’re actually running in your target instance.

How we built each of those stays in-house. But none of it is glamorous. It’s foundation data for using AI in development work, which if you’ve read anything I’ve written, you know I’m going to tell you to do first.

So before you let an AI write a single line of code, answer three questions:

Where does it get the data model, including your customizations?

Where does it get documentation that matches your release?

Who keeps both current?

If the answer to any of those is 'it just knows,' let me tell me you...it doesn’t.

In my next post we’ll get into now-sdk, and why building in source files changes what an agent can do for you.