- Post History
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
8 hours ago - edited 7 hours ago
Layer 2
The gatekeeper you can build today.
One narrow agent on every unqualified record. What it should actually do — judge quality, ask the right question back, answer the easy ones — and a concrete way to build it.
← Part 3: Layer 1 — Deflect Where the Request Is Born
Everything Layer 1 did not resolve or fully qualify arrives here. Layer 2 is a single horizontal agent that runs on every unqualified record, and its job is a decision rather than a resolution. It is the smallest thing in this entire series and, in our experience, the one with the best ratio of effort to effect.
Numbering, once, so it does not confuse anyone: this is Part 4 of the series and it covers the second of the three layers.
It is also the layer people try hardest to make bigger. Resist that. The moment the gatekeeper starts fixing things, it stops being auditable, its runtime cost stops being predictable, and nobody can say afterwards why a record went where it went. “Bigger” here means fulfilling, not sensing — a second, narrow agent that only extracts evidence the gatekeeper cannot read itself (a screenshot, say) and writes it to work notes is still inside the five outcomes below, because it decides nothing on its own. We come back to that distinction later in this article, once we had actually built one.
What the missing layer costs right now
You do not need a business case for this layer, you need a churn measurement. Two numbers from the starter set in Part 2 usually settle it on their own:
- Reassignments per record. Anything meaningfully above zero means work is being passed around before someone competent looks at it. Where customers have measured it with us, it sat around two thirds of a reassignment per incident — averaged across everything, including the ones that landed correctly first time.
- Field modifications before close. Consistently high single digits. That is people correcting what intake should have captured.
Add a third that is not about routing at all: the share of records that close with no close notes. Consistently around a third. Every one of those is a resolution that happened and then evaporated — nothing downstream, human or agent, can ever reuse it.
Triage is not a reporting problem, it is a work-allocation problem. The churn happens after the record exists, which is why better dashboards have never fixed it. Something has to make the decision at the moment of arrival.
Five outcomes, and nothing else
The gatekeeper has exactly five possible exits. Enumerating them is not documentation — it is the design. If a sixth behaviour appears in testing, that is the signal that scope has crept.
| Outcome | What happens | Writes |
|---|---|---|
| Auto-resolve | The record is a question with a known answer. Answer it from knowledge, with the source, and close it. | Resolution, close notes, source |
| Resolve directly | A known error with an approved workaround. Deliver the workaround and confirm it fits. | Resolution, close notes, KB link |
| Clarify, then route | The category is clear but a required fact is missing. Ask for exactly that one thing, in the same channel, and wait. | Category, comment, state |
| Categorise, hand off | Complete enough. Set the category that triggers the right vertical and get out of the way. | Category, assignment |
| Escalate | Outside the approved set, or confidence too low. Hand to a person with everything it found attached. | Work note only |
Note what is not in that list: provisioning, resetting, restarting, configuring, ordering. All of that is Layer 3. The gatekeeper never touches a system of record other than the ticket in front of it.
Four responsibilities. Never more. A tool budget is the most effective governance control we have found for this layer, because it is checkable in a review — just not always by counting raw tool records. Enrich, Retrieve, Clarify and Decide is the whole job. In our own build, taking “cheapest path first” below seriously fanned Retrieve out into four concrete tools on its own, and we landed at nine tools in total across those four responsibilities. That is still disciplined, because every one of the nine answers to exactly one of the four jobs. What would not be disciplined is a tenth tool that answers to none of them — that is the actual test, and it is still checkable by counting, just at the responsibility level rather than the raw tool count.
What “quality” means, precisely enough to build
“The agent should improve ticket quality” is the kind of requirement that produces a chatty assistant nobody wants. So define it.
Quality here is a single, testable property: can the receiving team start work without asking the requester anything? That is it. Not grammar, not length, not tone.
Which turns the whole problem into something concrete. Each approved category carries a short list of facts that must be present before work can start — a definition of ready. Not a form, not fifteen mandatory fields: two to four facts, decided by the people who actually do the work.
| Category (illustrative) | Required before it can be worked |
|---|---|
| Access request | Which system · which permission or role · for whom · business justification if elevated |
| Device failure | Which device (asset or serial) · what fails · since when · whether the user is blocked |
| Application error | Which application · the exact message · reproducible or one-off · how many people affected |
| Connectivity | Location · wired, wireless or remote · what is unreachable · since when |
Now the agent has a deterministic job instead of a stylistic one: classify, look up that category’s required facts, compare against what is present, and ask for what is missing. Nothing about that requires creativity, and all of it is inspectable.
Put the categories and their required facts in a table, not in the prompt. Two reasons. The business owner of the taxonomy can change a required fact without anyone editing an agent, and every change is versioned in the platform rather than buried in an instruction. This single decision is what makes the layer survive its second year.
Full disclosure: our own demo build has not done this yet. The required facts per category live in the agent’s own instructions, and only the target assignment group sits in a proper, admin-editable table. It works at demo scale precisely because we are the only ones editing it. It is the first thing on our list to fix before anyone else is allowed to own a category — which is exactly the failure mode this box warns about.
Asking back, without becoming annoying
Clarification is the behaviour most likely to be hated, so it needs the tightest rules. Ours, learned the hard way:
Never ask for something you can look up
If the caller’s department, location, manager, assigned laptop or primary phone is on the record or one hop away in the CMDB or user table, the agent resolves it silently. Asking a user for their own asset tag is the fastest way to teach an organisation that the agent is worse than the form it replaced. Enforce this in the tool, not in the instruction — the lookup tool should be called before the clarification tool can be.
Two rounds, hard cap, then out
Bundle everything missing for that category into a single message. We shipped this as strictly one round in an earlier draft of this piece. Building it changed our mind: a real reply to a real bundled question is often close but not quite complete — one more concrete fact, not a whole new interrogation — and cutting the conversation there felt like punishing the caller for trying. So we allow exactly one follow-up: if the second reply still does not complete the picture, do not ask a third time — categorise on what you have, note what is still open, and hand over. The discipline is the hard cap, not the round count; two rounds of genuine interrogation on a P4 request still costs more goodwill than the automation saves.
Ask in the channel the request came from
A clarification that arrives as an email notification on a request the user raised in chat will not be answered. If the originating channel is still open, ask there; if not, ask where the user will actually see it, and say what happens if they do not reply.
Put a clock on it
A record parked awaiting information is a record nobody owns. Define the timeout up front — after it expires, the agent categorises with what it has and routes. Never leave the ticket suspended indefinitely because a question went unanswered.
Answering the easy ones — carefully
This is the newest of the five outcomes and the one we were most cautious about, because it is the only exit where the agent tells a human something is finished.
The justification comes straight out of Layer 1’s search data: a large share of what lands as a record is not a request for work at all. It is a question with an existing answer — where something is, whether something is allowed, how something is done. Those records need no category, no assignment group and no vertical. They need an answer and a close.
Four conditions, all of which must hold:
- It is a question, not a failure report. “How do I request X” qualifies. “X is broken” never does, however well-documented the fix.
- There is a grounded source. A specific, current knowledge article — and the answer cites it. Anything the agent produced without a source goes to escalate instead.
- Retrieval confidence is above a threshold you set deliberately. Start high. You can always lower it once you have seen a month of decisions.
- There is a one-click way to say it did not help that reopens the record, marks it as a failed auto-resolve, and routes it to a human without making the user retype anything.
That fourth condition is the important one and it is usually the one that gets cut for time. Auto-resolve without a cheap escape hatch does not reduce work — it converts one ticket into one ticket plus one complaint. We are saying this as the people who cut it: our own demo build gates auto-resolve on a real knowledge article, but does not yet have the one-click reopen wired up. It is next, not optional — we would rather name the gap here than let it quietly become permanent.
Track failed auto-resolves as their own number from day one, not as part of a general reopen rate. It is the single best early-warning signal for this layer, and it tells you whether to move the confidence threshold up or down with evidence instead of instinct.
How we would build it
Concrete enough to start, generic enough to survive your instance being different. Everything below is standard platform capability — no new products to stand up.
To be clear about what this section is: a design, drawn from what we have built at demo scale. There is no package to import here and nothing production-hardened. Take it as a starting point you will improve on, not as a specification to follow literally.
The gatekeeper, end to end
One trigger, four responsibilities, five exits — and escalation as an audited exit, not a silent one.
|
Trigger
Record created with no or generic category. Exclude, explicitly · Machine-generated records · Already pre-qualified intake · Major incidents / above your priority line Cheapest path first: classifier before any generative call. |
→ |
Four responsibilities
Not necessarily four tools — nine, in our build. 1 Enrich Pull what is already knowable: caller, assets, similar records. Runs first. ↓
2 Retrieve Grounded candidate answer, with source. Fans out into a classifier, a similarity search and a KB search once you take “cheapest path first” seriously. ↓
3 Clarify One bundled question, in the originating channel. One follow-up allowed if still incomplete — hard cap, then hand over. ↓
4 Decide Writes the outcome. The only tool allowed to write, and only these fields. |
→ |
Five exits — and nothing else
Auto-resolve Answer a question from knowledge, with the source. Resolve directly Deliver an approved workaround. Clarify, then route Ask for the one missing required fact. Categorise, hand off Set the category that triggers the vertical. Escalate Hand to a person with everything it found attached. |
Escalate is a tool too
A fifth, constrained tool: it fires only once every other path has failed or does not apply. It can write a work note and a flag — never category, never assignment. A broken or uncertain agent still degrades into today’s behaviour; now it is an audited degrade, not a silent one. Every run writes
Outcome · category · evidence · source · confidence · what it chose not to do. |
The contract underneath
One table, one row per approved category: the required facts before work can start, the target vertical, whether auto-resolve is permitted — and a named business owner.
Build this before the agent. The build lands in weeks; the agreement is what paces the layer.
Figure 1. One trigger, four responsibilities, five exits — and the contract table underneath that everything else depends on.
1. The category contract table
A small custom table, one row per approved category. Fields: category, the two to four required facts, the target assignment group or vertical, whether auto-resolve is permitted for this category, and the business owner’s name. That last field is not decoration — a category with no named owner is exactly how routing taxonomies rot.
This table is the contract. Build it before you build the agent, and get it signed off by the people who receive the work. The build itself lands in weeks; what paces this layer is the agreement, and that is exactly why you start with it rather than waiting for it.
It is also the moment this stops being a technical project. Miriam from Part 1 does not need to approve a tool here — she needs to name, for every category, the person who owns what that category means. Nobody enjoys that conversation. Every organisation we have seen skip it has rebuilt this layer within a year.
And it is the one part Jonas cannot do for her. He can build every tool in this article; he cannot decide on the business’s behalf what “access request” is supposed to mean. If you are the Jonas in your organisation, this table is the thing worth escalating for.
2. The trigger, and what it must exclude
Fire on record creation where the category is empty or generic. Then exclude, explicitly:
- Machine-generated records — monitoring integrations, scheduled jobs, technical accounts. They are already structured; running a qualification agent over them burns budget to confirm what is already known.
- Records that arrived through a qualifying flow — anything Layer 1 already pre-qualified does not need qualifying twice.
- Major incidents and anything above your priority line. Those go to a human immediately. A gatekeeper adding thirty seconds of latency to a P1 is a defect, not a feature.
3. Cheapest path first
Before any generative call, try the deterministic route. A trained classification model over your historical records is fast, costs nothing per run, and on well-populated categories is genuinely good. Use it as the first pass and only invoke the language model when the classifier is not confident enough.
This is a cost decision and a quality decision at once. Most records in a mature instance are boring and repetitive, and the boring ones should never reach an expensive path. On the question of arbitrating between a classifier, retrieval and a generative call — including how to gate writes behind a confidence threshold — there is an excellent Community article on multi-method assignment group prediction that goes deeper than we will here. Read it alongside this one.
4. The four responsibilities (not necessarily four tools)
The table below names four responsibilities, and in a minimal build each is genuinely one tool. The moment you take “cheapest path first” seriously, though, Retrieve stops being one tool: a trained classifier, a keyword or semantic similarity lookup, and a knowledge-base search are three different calls with three different confidence signals, arbitrated the way the linked Community article describes. In our own build that put us at nine concrete tools across these four responsibilities — still auditable, because we can point at every one of the nine and say which of the four jobs it does.
| Tool | What it does | Type |
|---|---|---|
| Enrich | Pulls what is already knowable: caller details, department, assigned assets, open records on the same CI, recent similar records. Runs first, always. | Script / record lookup |
| Retrieve | Searches knowledge and the catalog for a grounded candidate answer, returning the source. Feeds both auto-resolve and the workaround path. | AI Search retrieval |
| Clarify | Posts one bundled question to the requester in the originating channel, sets the awaiting-info state, and starts the timeout. | Script / record update |
| Decide | Writes the outcome: category, assignment, or resolution with close notes and source. The only tool allowed to write a decision, and restricted to those fields. | Record update |
We originally designed Escalate as not-a-tool: the outcome that happens when nothing else fires, with the agent simply writing a note and stopping. Building it changed our mind on the mechanics, not the principle. A silent stop is indistinguishable, in a log, from a stuck or broken agent — and this whole layer is supposed to be reviewable. So Escalate is a fifth explicit tool in our build too, but a constrained one: it can only write a work note and a flag, it can never touch category or assignment, and every other path (auto-resolve, resolve, clarify, categorise) must actively fail or not apply before the agent is allowed to call it. The safety property the original design was protecting — a broken agent degrades into today’s behaviour, never into something worse — still holds. It is just an audited degrade now, not a silent one.
5. The instruction, in one paragraph
The instruction should read roughly: you are a triage agent; you classify and qualify, you never fulfil; the permitted categories and their required facts come from the contract table; if the record is complete, categorise and stop; if one required fact is missing, clarify once; if it is a question with a grounded answer and auto-resolve is permitted for that category, answer and close; in every other case escalate with a work note.
If your instruction is substantially longer than that, the scope is wrong. Length in a triage instruction is a symptom, not thoroughness.
6. Make every run explain itself
One work note per run, in a fixed shape: which outcome, which category, what evidence, which source, what confidence, what it chose not to do. The platform trace gives you the technical execution path; this note gives the service owner the human-readable version — and it is what turns “the AI moved my ticket” into a reviewable decision.
7. When a second agent is still Layer 2
We built a genuine exception to “one narrow agent” and want to be upfront about it. Some records carry an attachment — a screenshot of an error dialog, most often — that the gatekeeper’s own Enrich tool cannot read. We added a second, separately scoped agent whose only job is: pull the attachment, describe what it shows, write that description to the incident’s work notes, and stop. It never sets a category, never assigns, never resolves — it only makes the record readable, which is exactly what Enrich already does for text. An orchestrator dispatches it before the gatekeeper runs, unconditionally, because it is cheap to check and near-free to skip: if there is no attachment, it finishes in one tool call and does nothing else.
Why a second agent instead of a fifth tool on the gatekeeper itself: the attachment-analysis job is asynchronous and has to poll a task until it completes, retrying in a fallback mode on error — a multi-step loop with its own retry logic, not a single call and response. Our platform expresses that naturally as a small agent and awkwardly as one tool. If your platform can express an async poll-and-retry loop inside a single tool call, we would genuinely rather you did that and kept this at one agent — we only reached for a second one because we needed the control flow, not because the job deserved its own opinions. Either way, the rule holds: this second agent has zero decision authority over the five outcomes. It enriches; the gatekeeper still decides.
Before production
- Every category in the contract table has a named business owner and a documented downstream outcome.
- The agent has been run in read-only mode over a few hundred historical records and the proposed decisions reviewed by the receiving teams — before it is allowed to write anything.
- Write access is restricted to the fields in the Decide tool. Verify this by attempting something else and confirming it fails.
- Machine-generated and above-threshold records are provably excluded by the trigger, tested with real examples.
- Auto-resolve is off for every category except the ones explicitly enabled, and each of those has a current, owned knowledge article behind it.
- The awaiting-information timeout is defined, and the behaviour on expiry is implemented and tested.
- Four numbers are being recorded from day one: reassignments per record, share clarified, share auto-resolved, failed auto-resolves.
- Someone owns the weekly review of the escalate bucket. That bucket is your backlog of missing categories — if nobody reads it, the taxonomy never improves.
Not “the agent resolves 40% of tickets.” It should be: reassignments per record measurably down, the receiving teams reporting fewer ask-backs, and the escalate bucket shrinking as categories get added. If those three move, the layer is working — even if the resolution rate stays modest. That is what a gatekeeper is for.
Where this hands over
Everything above exists so that one thing is true downstream: a record leaving Layer 2 has a category that means something. A reliable, pre-approved category is the trigger a specialist agent can be built against. Without it, every Layer 3 agent has to re-derive what kind of work it is looking at, which is both wasteful and the reason vertical agents so often feel unpredictable.
Part 5 takes that up: specialists scoped to a category rather than to a team, why the org chart is the wrong boundary, and how to pick which category to build for first. It publishes shortly, together with the closing part on what bites you when you build any of this.
The series
| Part | Topic | Status |
|---|---|---|
| 01 | One flow, three layers — the pattern | Read now |
| 02 | Read the instance before you design the agent — harvest, filter, score, visualise | Read now |
| 03 | Layer 1 — deflect where the request is born, including a working build to import | Read now |
| 04 | Layer 2 — the gatekeeper you can build today | You are here |
| 05 | Layer 3 — specialists scoped to a category, not a team | Next week |
| 06 | Six things that bite you when you build this | Next week |
Tell us where we are wrong
Particularly on two things, both of which moved since we first drafted this piece. We said we had ruled out a second clarification round on judgement rather than data — then built the thing and allowed exactly one follow-up, for the reasons in Rule 02. And we said three to four tools, then landed on nine once real assignment-group prediction and a second, narrow enrichment agent were both in the mix. If your own build kept either number lower than ours and it still holds up under real traffic, we would like to know what you left out that we did not.
Views are our own and do not represent our team, employer, partners, or customers. Anything we build and share in this series is a demo-grade MVP — not a ServiceNow product, not part of any roadmap, and not supported. Category names, required facts and thresholds in this article are illustrative — yours come from your own taxonomy and your own review.
