- Post History
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
3 weeks ago - edited 3 weeks ago
Layer 2
The gatekeeper you can build today.
One narrow agent on every unqualified record. What it should actually do — judge quality, ask the right question back, answer the easy ones — and a concrete way to build it.
← Part 3: Layer 1 — Deflect Where the Request Is Born
Everything Layer 1 did not resolve or fully qualify arrives here. Layer 2 is a single horizontal agent that runs on every unqualified record, and its job is a decision rather than a resolution. It is the smallest thing in this entire series and, in our experience, the one with the best ratio of effort to effect.
Numbering, once, so it does not confuse anyone: this is Part 4 of the series and it covers the second of the three layers.
It is also the layer people try hardest to make bigger. Resist that. The moment the gatekeeper starts fixing things, it stops being auditable, its runtime cost stops being predictable, and nobody can say afterwards why a record went where it went. “Bigger” here means fulfilling, not sensing — a second, narrow agent that only extracts evidence the gatekeeper cannot read itself (a screenshot, say) and writes it to work notes is still inside the five outcomes below, because it decides nothing on its own. We come back to that distinction later in this article, now that we have actually built one.
What the missing layer costs right now
You do not need a business case for this layer, you need a churn measurement. Two numbers from the starter set in Part 2 usually settle it on their own:
- Reassignments per record. Anything meaningfully above zero means work is being passed around before someone competent looks at it. Where customers have measured it with us, it sat around two thirds of a reassignment per incident — averaged across everything, including the ones that landed correctly first time.
- Field modifications before close. Consistently high single digits. That is people correcting what intake should have captured.
Add a third that is not about routing at all: the share of records that close with no close notes. Consistently around a third. Every one of those is a resolution that happened and then evaporated — nothing downstream, human or agent, can ever reuse it.
Triage is not a reporting problem, it is a work-allocation problem. The churn happens after the record exists, which is why better dashboards have never fixed it. Something has to make the decision at the moment of arrival.
Five outcomes, and nothing else
The gatekeeper has exactly five possible exits. Enumerating them is not documentation — it is the design. If a sixth behaviour appears in testing, that is the signal that scope has crept.
| Outcome | What happens | Writes |
|---|---|---|
| Auto-resolve | The record is a question with a known answer. Answer it from knowledge, with the source, and close it. | Resolution, close notes, source |
| Resolve directly | A known error with an approved workaround. Deliver the workaround and confirm it fits. | Resolution, close notes, KB link |
| Clarify, then route | The category is clear but a required fact is missing. Ask for exactly that one thing, in the same channel, and wait. | Category, comment, state |
| Categorise, hand off | Complete enough. Set the category that triggers the right vertical and get out of the way. | Category, assignment |
| Escalate | Outside the approved set, or confidence too low. Hand to a person with everything it found attached. | Work note only |
Note what is not in that list: provisioning, resetting, restarting, configuring, ordering. All of that is Layer 3. The gatekeeper never touches a system of record other than the ticket in front of it.
Four responsibilities: enrich, retrieve, clarify, decide. That is the whole job of this layer. How many actual tools that becomes depends on your platform and how far you take “cheapest path first” — ours ended up as more than four, and that is fine. The test is not the count. It is whether you can point at every tool and say which of the four jobs it answers to. A tool that answers to none of them is scope creep wearing a useful name.
What “quality” means, precisely enough to build
“The agent should improve ticket quality” is the kind of requirement that produces a chatty assistant nobody wants. So define it.
Quality here is a single, testable property: can the receiving team start work without asking the requester anything? That is it. Not grammar, not length, not tone.
Which turns the whole problem into something concrete. Each approved category carries a short list of facts that must be present before work can start — a definition of ready. Not a form, not fifteen mandatory fields: two to four facts, decided by the people who actually do the work.
| Category (illustrative) | Required before it can be worked |
|---|---|
| Access request | Which system · which permission or role · for whom · business justification if elevated |
| Device failure | Which device (asset or serial) · what fails · since when · whether the user is blocked |
| Application error | Which application · the exact message · reproducible or one-off · how many people affected |
| Connectivity | Location · wired, wireless or remote · what is unreachable · since when |
Now the agent has a deterministic job instead of a stylistic one: classify, look up that category’s required facts, compare against what is present, and ask for what is missing. Nothing about that requires creativity, and all of it is inspectable.
Put the categories and their required facts in a table, not in the prompt. Two reasons. The business owner of the taxonomy can change a required fact without anyone editing an agent, and every change is versioned in the platform rather than buried in an instruction. This single decision is what makes the layer survive its second year.
Full disclosure: our own demo build has not done this yet. The required facts per category live in the agent’s own instructions, and only the target assignment group sits in a proper, admin-editable table. It works at demo scale precisely because we are the only ones editing it. It is the first thing on our list to fix before anyone else is allowed to own a category — which is exactly the failure mode this box warns about.
Asking back, without becoming annoying
Clarification is the behaviour most likely to be hated, so it needs the tightest rules. Ours, learned the hard way:
Never ask for something you can look up
If the caller’s department, location, manager, assigned laptop or primary phone is on the record or one hop away in the CMDB or user table, the agent resolves it silently. Asking a user for their own asset tag is the fastest way to teach an organisation that the agent is worse than the form it replaced. Enforce this in the tool, not in the instruction — the lookup tool should be called before the clarification tool can be.
One question per run, and count them
Bundle everything missing for that category into a single message, then stop. Do not hold a conversation: park the record as awaiting caller information and end the run. When the reply arrives, that is a new run, and it re-evaluates the record from scratch.
Which is where the trap is, and we walked into it. The loop is stateless. If the reply still leaves a required fact missing, the same “ask once” rule fires again — legitimately, because the new run knows nothing about the previous one. “Ask once” quietly becomes “ask once per run”, and a P4 request can end up in a polite ping-pong nobody designed.
So the cap does not belong in the rule, it belongs on the record: count how many clarifications this record has already had. After the cap, categorise on what you have, note what is still open, and hand over. Two rounds of genuine interrogation on a P4 request still costs more goodwill than the automation saves — but that is now enforced by a counter, not by an instruction the agent cannot remember.
Ask in the channel the request came from
A clarification that arrives as an email notification on a request the user raised in chat will not be answered. If the originating channel is still open, ask there; if not, ask where the user will actually see it, and say what happens if they do not reply.
Put a clock on it
A record parked awaiting information is a record nobody owns. Define the timeout up front — after it expires, the agent categorises with what it has and routes. Never leave the ticket suspended indefinitely because a question went unanswered.
Answering the easy ones — carefully
This is the newest of the five outcomes and the one we were most cautious about, because it is the only exit where the agent tells a human something is finished.
The justification comes straight out of Layer 1’s search data: a large share of what lands as a record is not a request for work at all. It is a question with an existing answer — where something is, whether something is allowed, how something is done. Those records need no category, no assignment group and no vertical. They need an answer and a close.
Four conditions, all of which must hold:
- It is a question, not a failure report. “How do I request X” qualifies. “X is broken” never does, however well-documented the fix.
- There is a grounded source. A specific, current knowledge article — and the answer cites it. Anything the agent produced without a source goes to escalate instead.
- Retrieval confidence is above a threshold you set deliberately. Start high. You can always lower it once you have seen a month of decisions.
- There is a one-click way to say it did not help that reopens the record, marks it as a failed auto-resolve, and routes it to a human without making the user retype anything.
That fourth condition is the important one and it is usually the one that gets cut for time. Auto-resolve without a cheap escape hatch does not reduce work — it converts one ticket into one ticket plus one complaint. We are saying this as the people who cut it: our own demo build gates auto-resolve on a real knowledge article, but does not yet have the one-click reopen wired up. It is next, not optional — we would rather name the gap here than let it quietly become permanent.
Track failed auto-resolves as their own number from day one, not as part of a general reopen rate. It is the single best early-warning signal for this layer, and it tells you whether to move the confidence threshold up or down with evidence instead of instinct.
How we would build it
Concrete enough to start, generic enough to survive your instance being different. Everything below is standard platform capability — no new products to stand up.
To be clear about what this section is: one way to do it, not the way. It is drawn from what we built at demo scale, which means it carries our platform, our constraints and a couple of choices we would make differently now. There is no package to import here and nothing production-hardened.
Read it as a worked example. If your shape ends up different — fewer moving parts, more, or a completely different split — that is not a deviation from something. The five outcomes and the category contract are the parts we would actually defend. Everything below them is implementation detail, and implementation detail is supposed to differ.
The gatekeeper, end to end
One trigger, four responsibilities, five exits — and escalation as an audited exit, not a silent one.
|
Trigger
Record created with no or generic category. Exclude, explicitly · Machine-generated records · Already pre-qualified intake · Major incidents / above your priority line Cheapest path first: classifier before any generative call. |
→ |
Four responsibilities
Not necessarily four tools — nine, in our build. 1 Enrich Pull what is already knowable: caller, assets, similar records. Runs first. ↓
2 Retrieve Grounded candidate answer, with source. Fans out into a classifier, a similarity search and a KB search once you take “cheapest path first” seriously. ↓
3 Clarify One bundled question, in the originating channel, then park and end the run. A counter on the record caps how often this may repeat. ↓
4 Decide Writes the outcome. The only tool allowed to write, and only these fields. |
→ |
Five exits — and nothing else
Auto-resolve Answer a question from knowledge, with the source. Resolve directly Deliver an approved workaround. Clarify, then route Ask for the one missing required fact. Categorise, hand off Set the category that triggers the vertical. Escalate Hand to a person with everything it found attached. |
Escalate is a tool too
A fifth, constrained tool: it fires only once every other path has failed or does not apply. It can write a work note and a flag — never category, never assignment. A broken or uncertain agent still degrades into today’s behaviour; now it is an audited degrade, not a silent one. Every run writes
Outcome · category · evidence · source · confidence · what it chose not to do. |
The contract underneath
One table, one row per approved category: the required facts before work can start, the target vertical, whether auto-resolve is permitted — and a named business owner.
Build this before the agent. The build lands in weeks; the agreement is what paces the layer.
Figure 1. One trigger, four responsibilities, five exits — and the contract table underneath that everything else depends on.
0. You do not have to start from scratch
Before any of the steps below: a good deal of what this layer needs already ships. The workflow collections come with prebuilt AI agents that categorise a record, identify the affected configuration item from the caller’s own assets, surface similar records, retrieve knowledge, generate resolution notes and link major incidents. An orchestrator already knows how to choose between them.
So there are three legitimate starting points, and none of them is more correct than the others:
- Turn on what ships and stop there. Enable the prebuilt triage and categorisation agents, watch what they conclude for a few weeks, and see how far that gets you. For many organisations this is already most of Layer 2, and it costs a configuration decision rather than a project.
- Mix. Keep the prebuilt agents for the things they do well and add your own where your process is genuinely specific — which, in our experience, is the category contract and the clarification behaviour, not the classification.
- Build it yourself. Which is what the rest of this section describes, because that is what we did — mostly so we could see every moving part.
Whichever route you take, two things are never in the box, because nobody can ship them for you: the approved category set and who owns each category, and the discipline about escalation and measurement. If your Layer 2 plan is mostly engineering, it is pointed at the easier half of the problem.
1. The category contract table
A small custom table, one row per approved category. Fields: category, the two to four required facts, the target assignment group or vertical, whether auto-resolve is permitted for this category, and the business owner’s name. That last field is not decoration — a category with no named owner is exactly how routing taxonomies rot.
This table is the contract. Build it before you build the agent, and get it signed off by the people who receive the work. The build itself lands in weeks; what paces this layer is the agreement, and that is exactly why you start with it rather than waiting for it.
It is also the moment this stops being a technical project. Miriam from Part 1 does not need to approve a tool here — she needs to name, for every category, the person who owns what that category means. Nobody enjoys that conversation. Every organisation we have seen skip it has rebuilt this layer within a year.
And it is the one part Jonas cannot do for her. He can build every tool in this article; he cannot decide on the business’s behalf what “access request” is supposed to mean. If you are the Jonas in your organisation, this table is the thing worth escalating for.
2. The trigger, and what it must exclude
Fire on record creation where the category is empty or generic. Then exclude, explicitly:
- Machine-generated records — monitoring integrations, scheduled jobs, technical accounts. They are already structured; running a qualification agent over them burns budget to confirm what is already known.
- Records that arrived through a qualifying flow — anything Layer 1 already pre-qualified does not need qualifying twice.
- Major incidents and anything above your priority line. Those go to a human immediately. A gatekeeper adding thirty seconds of latency to a P1 is a defect, not a feature.
3. Cheapest path first
Before any generative call, try the deterministic route. A trained classification model over your historical records is fast, costs nothing per run, and on well-populated categories is genuinely good. Use it as the first pass and only invoke the language model when the classifier is not confident enough.
This is a cost decision and a quality decision at once. Most records in a mature instance are boring and repetitive, and the boring ones should never reach an expensive path. On the question of arbitrating between a classifier, retrieval and a generative call — including how to gate writes behind a confidence threshold — there is an excellent Community article on multi-method assignment group prediction that goes deeper than we will here. Read it alongside this one.
4. The four responsibilities (not necessarily four tools)
Four responsibilities, and in a minimal build each is genuinely one tool. Take “cheapest path first” seriously and Retrieve stops being one: a classifier, a similarity lookup and a knowledge search are three calls with three different confidence signals. Our own build ended up with more tools than responsibilities for exactly that reason. The number is not the point — being able to say which of the four jobs each one answers to is.
| Responsibility | What it does | Type |
|---|---|---|
| Enrich | Pulls what is already knowable: caller details, department, assigned assets, open records on the same CI, recent similar records. Runs first, always. | Script / record lookup |
| Retrieve | Searches knowledge and the catalog for a grounded candidate answer, returning the source. Feeds both auto-resolve and the workaround path. | AI Search retrieval |
| Clarify | Posts one bundled question to the requester in the originating channel, sets the awaiting-info state, and starts the timeout. | Script / record update |
| Decide | Writes the outcome: category, assignment, or resolution with close notes and source. The only tool allowed to write a decision, and restricted to those fields. | Record update |
One thing we would flag rather than prescribe. We first designed Escalate as not-a-tool: the outcome that happens when nothing else fires, with the agent simply writing a note and stopping. Building it changed our mind on the mechanics, not the principle. A silent stop is indistinguishable, in a log, from a stuck or broken agent — and this whole layer is supposed to be reviewable. So Escalate is a fifth explicit tool in our build too, but a constrained one: it can only write a work note and a flag, it can never touch category or assignment, and every other path (auto-resolve, resolve, clarify, categorise) must actively fail or not apply before the agent is allowed to call it. The safety property the original design was protecting — a broken agent degrades into today’s behaviour, never into something worse — still holds. It is just an audited degrade now, not a silent one.
5. The instruction, in one paragraph
The instruction should read roughly: you are a triage agent; you classify and qualify, you never fulfil; the permitted categories and their required facts come from the contract table; if the record is complete, categorise and stop; if one required fact is missing, clarify once; if it is a question with a grounded answer and auto-resolve is permitted for that category, answer and close; in every other case escalate with a work note.
If your instruction is substantially longer than that, the scope is wrong. Length in a triage instruction is a symptom, not thoroughness.
6. Make every run explain itself
One work note per run, in a fixed shape: which outcome, which category, what evidence, which source, what confidence, what it chose not to do. The platform trace gives you the technical execution path; this note gives the service owner the human-readable version — and it is what turns “the AI moved my ticket” into a reviewable decision.
7. When a second agent is still Layer 2
We built a genuine exception to “one narrow agent” and want to be upfront about it. Some records carry an attachment — a screenshot of an error dialog, most often — that the gatekeeper’s own Enrich tool cannot read. We added a second, separately scoped agent whose only job is: pull the attachment, describe what it shows, write that description to the incident’s work notes, and stop. It never sets a category, never assigns, never resolves — it only makes the record readable, which is exactly what Enrich already does for text. An orchestrator dispatches it before the gatekeeper runs, unconditionally, because it is cheap to check and near-free to skip: if there is no attachment, it finishes in one tool call and does nothing else.
Why a second agent rather than another tool on the gatekeeper was a control-flow question for us, not a design principle — the job is asynchronous and needs its own retry loop, which our setup expresses more naturally as a small agent. Do it however your platform prefers. The rule is the part that matters: this second agent has zero decision authority over the five outcomes. It enriches; the gatekeeper still decides.
Before production
- Every category in the contract table has a named business owner and a documented downstream outcome.
- The agent has been run in read-only mode over a few hundred historical records and the proposed decisions reviewed by the receiving teams — before it is allowed to write anything.
- Write access is restricted to the fields in the Decide tool. Verify this by attempting something else and confirming it fails.
- Machine-generated and above-threshold records are provably excluded by the trigger, tested with real examples.
- Auto-resolve is off for every category except the ones explicitly enabled, and each of those has a current, owned knowledge article behind it.
- The awaiting-information timeout is defined, and the behaviour on expiry is implemented and tested.
- Four numbers are being recorded from day one: reassignments per record, share clarified, share auto-resolved, failed auto-resolves.
- Someone owns the weekly review of the escalate bucket. That bucket is your backlog of missing categories — if nobody reads it, the taxonomy never improves.
Not “the agent resolves 40% of tickets.” It should be: reassignments per record measurably down, the receiving teams reporting fewer ask-backs, and the escalate bucket shrinking as categories get added. If those three move, the layer is working — even if the resolution rate stays modest. That is what a gatekeeper is for.
Where this hands over
Everything above exists so that one thing is true downstream: a record leaving Layer 2 has a category that means something. A reliable, pre-approved category is the trigger a specialist agent can be built against. Without it, every Layer 3 agent has to re-derive what kind of work it is looking at, which is both wasteful and the reason vertical agents so often feel unpredictable.
Part 5 takes that up: specialists scoped to a category rather than to a team, why the org chart is the wrong boundary, and how to pick which category to build for first. It publishes shortly, together with the closing part on what bites you when you build any of this.
The series
| Part | Topic | Status |
|---|---|---|
| 01 | One flow, three layers — the pattern | Read now |
| 02 | Read the instance before you design the agent — harvest, filter, score, visualise | Read now |
| 03 | Layer 1 — deflect where the request is born, including a working build to import | Read now |
| 04 | Layer 2 — the gatekeeper you can build today | You are here |
| 05 | Layer 3 — specialists scoped to a category, not a team | Next week |
| 06 | Six things that bite you when you build this | Next week |
Tell us where we are wrong
Two things in particular. If you started from the prebuilt agents and never needed to build your own gatekeeper, we would like to hear how far that got you — that is the route we suspect most organisations should take and the one we have the least data on. And if you run a sixth outcome in production without it quietly turning into a fulfilment agent, tell us what it is, because we could not find one.
Views are our own and do not represent our team, employer, partners, or customers. Anything we build and share in this series is a demo-grade MVP — not a ServiceNow product, not part of any roadmap, and not supported. Category names, required facts and thresholds in this article are illustrative — yours come from your own taxonomy and your own review.
