AI Guardian

AI Guardian is built on the ServiceNow Small Language Model (SLM) and monitors generative AI interactions to detect offensive content, prompt injection attacks, and sensitive topics.

AI Guardian Overview

Generative AI is an emerging technology. Human interactions are unpredictable, and outputs generated by large language model (LLM) are probabilistic, which means that they're based on probabilities. Running the same input twice may generate two different outputs. Managing this risk is an important consideration when implementing generative AI on your instance. AI Guardian evaluates requests sent to LLMs and their responses in real-time to reduce that risk.

Guardrails

AI Guardian provides three guardrails. Each guardrail has a different scope of applicability:

Guardrail	What it detects	Scope
Offensiveness detection	Offensive or harmful content in AI inputs and outputs.	Specific Now Assist skills and workflows.
Prompt injection detection	Attempts to override LLM instructions or expose restricted information.	All generative AI applications and features. You can configure prompt injection detection at the instance level or for individual skills.
Sensitive topic filters	Subjects not suited for AI responses, such as workplace safety or employee compensation.	Virtual Agent conversational skills only (available for HR Service Delivery and Customer Service Management).

Note:

The scope of each guardrail differs. Prompt injection detection applies to all generative AI applications and features. Offensiveness detection applies only to supported Now Assist skills and workflows. Sensitive topic filters apply only to Virtual Agent conversations and require HR Service Delivery.

Offensive content

Due to the probabilistic nature of generative AI, it's possible for an LLM to generate offensive content. If there's offensive content in the input of the request, offensive content can also occur in the response. Examples of offensive content include language that is toxic, defamatory, or fraudulent.

When offensive content is detected, AI Guardian logs the event by default. You can also configure it to block the content. This guardrail applies to specific Now Assist skills and workflows.

Prompt injection

Prompt injection is a type of security attack where someone tries to override the normal instructions of an LLM to access restricted information or cause unintended behaviors. AI Guardian detects prompt injection attempts by using an LLM trained on various types of prompt injection techniques, such as role playing, paraphrasing, repetition, instructions to ignore other instructions, and persuasion.

Note:

Due to the probabilistic nature of the model and evolving attack techniques, AI Guardian may not identify every prompt injection attempt in some cases.

Prompt injection protection applies to all generative AI applications and features on your instance. You can configure it at the instance level or for individual skills. When a skill has its own setting, Now Assist Guardian automatically applies the more protective of the two settings, the skill-level setting or the instance-level setting.

Filtered subjects

Certain subjects, such as workplace safety employee compensation, or personal well-being may not be best suited for generative AI responses. You can activate filters that detect these kinds of subjects in Virtual Agent conversations and redirect users to the Sensitivity Detection: Fallback Virtual Agent topic instead of generating an AI response.

Note:

Sensitive topic filters apply only to Virtual Agent conversational skills. These filters are available only with HR Service Delivery and Customer Service Management.

Logging and blocking

AI Guardian detects and logs events for offensive content and prompt injection attempts by default. You can access logs from Now Assist Admin > Settings > Now Assist Guardian. Log data includes information about the request, the conversation that contains the offensive content, and any user feedback.

In addition to logging, you can configure AI Guardian to block offensive content or prompt injection attempts. When blocking is enabled and content is detected, you see a standard error message instead of the generated response. The standard error message displays that the request couldn’t be completed, and you don't see what the AI generated. Before enabling blocking, review logs for a period of time to understand how frequently these issues occur in your environment.

Redirection for sensitive filtered topics

After a filter detects a sensitive topic, AI Guardian redirects you to the Sensitivity Detection: Fallback topic in Virtual Agent. This topic can redirect you to a live agent or help you create an HR case.

You can override the redirection by selecting Proceed, not sensitive. It returns you to their original topic without initiating the fallback flow.

Note:

After you continue with the fallback topic, for example, by starting the flow to create an HR case, Virtual Agent does not continue detecting sensitive topics in that conversation.

AI Guardian at runtime

All skills that use AI Guardian remove personally identifiable information (PII) before the request reaches the LLM. You can configure what type of data is anonymized. For more information see, Configuring Now Assist for Data Privacy.

For conversational skills, semantic search processes requests to determine whether a filter has been detected. If so, the user is redirected to a Virtual Agent topic that asks if they want to make an HR case or speak to a live agent.

For catalog item generation and agent skills, such as summarization and resolution note generation, offensiveness and prompt injection guardrails run on inputs and outputs of requests. If either is detected, AI Guardian logs the request. If you’ve chosen to block this content, then a standard error message appears, and the user doesn’t see the result of the request.

Safety categories captured in AI Guardian logs

The following are the different kinds of content that can be flagged as offensive or poses a security risk. You can identify what categories are detected in the AI Guardian logs for the different guardrails.

Category	ID	Description
Violent Crimes	S1	Should not: Enable, encourage, or endorse unlawful violence toward people or animals, including murder, assault, torture, kidnapping, and animal abuse. Can: Explain criminal justice systems, lawful reporting procedures, or home emergency preparedness.
Non-Violent Crimes	S2	Should not: Enable, encourage, or endorse non-violent crimes, including cybercrime, hacking, fraud, theft, drug trafficking, and money laundering. Can: Explain cybersecurity defense techniques, how systems work, or technical concepts like HTTP vs HTTPS.
Sex-Related Crimes	S3	Should not: Enable, encourage, or endorse sex-related crimes, including sexual assault, rape, sexual coercion, sexual harassment, sex trafficking, and prostitution-related offenses. Can: Provide information about victim support services, domestic violence shelters, or lawful reporting procedures.
Child Sexual Exploitation	S4	Should not: Enable, encourage, or endorse child sexual abuse, including grooming, solicitation of explicit material, and child sexual abuse material. Can: Provide online safety education for parents or discuss unrelated academic topics.
Specialized Advice	S5	Should not: Provide personalized medical diagnosis, case-specific legal strategy, individualized financial advice, or falsely assert that dangerous activities are safe without professional oversight. Can: Discuss general medical information, symptoms, or treatment programs.
Privacy	S6	Should not: Contain sensitive personal information, including account credentials, government identifiers, financial details, location data, or medical records. Can: Discuss privacy policies and regulations, technical setup, or unrelated general tasks.
Intellectual Property	S7	Should not: Reproduce copyrighted works, lyrics, scripts, pirated software, or leak proprietary materials. Can: Discuss intellectual property law, write original content, or discuss research methodology.
Indiscriminate Weapons	S8	Should not: Enable, encourage, or endorse creation of chemical, biological, radiological, nuclear, or high-yield explosive weapons. Can: Discuss nonproliferation treaties, firearm safety, or CPR procedures.
Hate	S9	Should not: Demean or dehumanize people based on race, ethnicity, religion, disability, sexual orientation, or gender identity. Can: Discuss history of hate speech laws, free speech principles, or cultural topics.
Suicide & Self-Harm	S10	Should not: Enable, encourage, or endorse suicide, self-injury, eating disorders, or harm to others. Can: Provide warning sign information, medical treatment approaches, or unrelated scientific topics.
Sexual Content	S11	Should not: Display sexually explicit material or content inappropriate for minors, including pornography, erotic chat, and sexual activity. Can: Create or discuss general unrelated content.
Jailbreak Prompts / Prompt Injection	S12	Should not: Attempt to override safety instructions through direct commands, jailbreaks, or extraction attempts. Can: Discuss AI ethics, content moderation systems, or scientific principles.

Skills that support offensive detection

Table 1. Supported skills by workflow

Workflow	Application	Supported skills
Technology	Now Assist for Configuration Management Database (CMDB)	Configuration item (CI) summarization Manage duplicate CIs Service Graph Connector diagnosis
Technology	Now Assist for IT Operations Management (ITOM)	Alert analysis Alert investigation
Technology	Now Assist for IT Service Management (ITSM)	Change request risk explanation Change request summarization Chat reply recommendation Chat summarization Incident assist Incident summarization KB generation Resolution notes generation Sidebar discussion summarization
Technology	Now Assist for Security Incident Response	Post-incident analysis Resolution notes generation Security incident recommended actions Security incident summarization
Technology	Now Assist for Strategic Portfolio Management (SPM)	Multi feedback summarization Planning item doc summarization Project doc summarization Project summary emails Story generation Write planning item
Customer	Now Assist for Customer Service Management (CSM)	Case summarization Chat recommendation Chat summarization Email recommendation KB generation Resolution notes generation Sidebar summarization
Customer	Now Assist for Field Service Management (FSM)	KB generation Sidebar summarization Work order task summarization
Customer	Now Assist for Financial Services Operations (FSO)	Case summarization Disputes intake via Virtual Agent
Customer	Now Assist for Public Sector Digital Services (PSDS)	Government case summarization Chat summarization
Employee	Now Assist for Health and Safety	Incident summarization
Employee	Now Assist for HR Service Delivery (HRSD)	Case summarization Chat summarization KB generation Resolution notes generation
Employee	Now Assist for Legal Service Delivery (LSD)	Legal request summarization
Employee	Now Assist in Contract Management	Contract analysis Contract metadata extraction
Creator	Now Assist for Creator	Catalog item generation
Finance & Supply Chain	Now Assist for Accounts Payable Operations (APO)	Record summarization
Finance & Supply Chain	Now Assist for Supplier Lifecycle Operations (SLO)	Supplier case summarization
Finance & Supply Chain	Now Assist for Sourcing and Procurement Operations (SPO)	Record summarization

• Activate offensiveness protection for generative AI
  Activate offensiveness detection to log or block offensive content generated by Now Assist skills and workflows.
• Configure prompt injection attack protection
  Activate or deactivate prompt injection attack detection settings to protect all generative AI interactions on your instance from malicious inputs and unintended model behaviors.
• Configure sensitive topic filters
  Set up filters in AI Guardian to redirect Virtual Agent users to a live agent or HR case when a sensitive subject is detected in a conversation.
• Export AI Guardian logs
  Export logs from AI Guardian to get insights into how often different guardrails are being detected and used.
• Configuring a Guardrail Service Provider
  AI Guardian supports default guardrails and third-party or custom guardrail service providers to extend AI content monitoring with your organization's AI governance policies.