Now Assist Guardian

Now Assist Guardian is built on the ServiceNow Small Language Model (SLM) and monitors generative AI interactions to detect offensive content, prompt injection attacks, and sensitive topics.

Now Assist Guardian Overview

Generative AI is an emerging technology. Human interactions are unpredictable, and outputs generated by large language model (LLM) are probabilistic, which means that they're based on probabilities. Running the same input twice may generate two different outputs. Managing this risk is an important consideration when implementing generative AI on your instance. Now Assist Guardian evaluates requests sent to LLMs and their responses in real-time to reduce that risk.

Guardrails

Now Assist Guardian provides three guardrails. Each guardrail has a different scope of applicability:

Guardrail	What it detects	Scope
Offensiveness detection	Offensive or harmful content in AI inputs and outputs.	Specific Now Assist skills and workflows.
Prompt injection detection	Attempts to override LLM instructions or expose restricted information.	All generative AI applications and features. You can configure prompt injection detection at the instance level or for individual skills.
Sensitive topic filters	Subjects not suited for AI responses, such as workplace safety or employee compensation.	Virtual Agent conversational skills only (available for HR Service Delivery and Customer Service Management).

Note:

The scope of each guardrail differs. Prompt injection detection applies to all generative AI applications and features. Offensiveness detection applies only to supported Now Assist skills and workflows. Sensitive topic filters apply only to Virtual Agent conversations and require HR Service Delivery.

Offensive content

Due to the probabilistic nature of generative AI, it's possible for an LLM to generate offensive content. If there's offensive content in the input of the request, offensive content can also occur in the response. Examples of offensive content include language that is toxic, defamatory, or fraudulent.

When offensive content is detected, Now Assist Guardian logs the event by default. You can also configure it to block the content. This guardrail applies to specific Now Assist skills and workflows.

Prompt injection

Prompt injection is a type of security attack where someone tries to override the normal instructions of an LLM to access restricted information or cause unintended behaviors. Now Assist Guardian detects prompt injection attempts by using an LLM trained on various types of prompt injection techniques, such as role playing, paraphrasing, repetition, instructions to ignore other instructions, and persuasion.

Note:

Due to the probabilistic nature of the model and evolving attack techniques, Now Assist Guardian may not identify every prompt injection attempt in some cases.

Prompt injection protection applies to all generative AI applications and features on your instance. You can configure it at the instance level or for individual skills. When a skill has its own setting, Now Assist Guardian automatically applies the more protective of the two settings, the skill-level setting or the instance-level setting.

Filtered subjects

Certain subjects, such as workplace safety employee compensation, or personal well-being may not be best suited for generative AI responses. You can activate filters that detect these kinds of subjects in Virtual Agent conversations and redirect users to the Sensitivity Detection: Fallback Virtual Agent topic instead of generating an AI response.

Note:

Sensitive topic filters apply only to Virtual Agent conversational skills. These filters are available only with HR Service Delivery and Customer Service Management.

Logging and blocking

Now Assist Guardian detects and logs events for offensive content and prompt injection attempts by default. You can access logs from Now Assist Admin > Settings > Now Assist Guardian. Log data includes information about the request, the conversation that contains the offensive content, and any user feedback.

In addition to logging, you can configure Now Assist Guardian to block offensive content or prompt injection attempts. When blocking is enabled and content is detected, you see a standard error message instead of the generated response. The standard error message displays that the request couldn’t be completed, and you don't see what the AI generated. Before enabling blocking, review logs for a period of time to understand how frequently these issues occur in your environment.

Redirection for sensitive filtered topics

After a filter detects a sensitive topic, Now Assist Guardian redirects you to the Sensitivity Detection: Fallback topic in Virtual Agent. This topic can redirect you to a live agent or help you create an HR case.

You can override the redirection by selecting Proceed, not sensitive. It returns you to their original topic without initiating the fallback flow.

Note:

After you continue with the fallback topic, for example, by starting the flow to create an HR case, Virtual Agent does not continue detecting sensitive topics in that conversation.

Now Assist Guardian at runtime

All skills that use Now Assist Guardian remove personally identifiable information (PII) before the request reaches the LLM. You can configure what type of data is anonymized. For more information see, Configuring Now Assist for Data Privacy.

For conversational skills, semantic search processes requests to determine whether a filter has been detected. If so, the user is redirected to a Virtual Agent topic that asks if they want to make an HR case or speak to a live agent.

For catalog item generation and agent skills, such as summarization and resolution note generation, offensiveness and prompt injection guardrails run on inputs and outputs of requests. If either is detected, Now Assist Guardian logs the request. If you’ve chosen to block this content, then a standard error message appears, and the user doesn’t see the result of the request.

Safety categories captured in Now Assist Guardian logs

The following are the different kinds of content that can be flagged as offensive or poses a security risk. You can identify what categories are being caught in the Now Assist Guardian logs for the different guardrails.

Category	Category ID	Description
Toxic Content	O1	Should not: Promote hate speech, incite violence like child abuse or bullying or engage in personal attacks or harassment. Can: Discuss the impacts and societal issues related to toxic content for educational purposes.
Unfair Representation	O2	Should not: Assist in perpetuating stereotypes or encourage discrimination or bias. Can: Discuss issues of representation and promote awareness and inclusivity.
Adult Content	O3	Should not: Display sexually explicit material or content inappropriate for minors including pornography, erotic chat and sexual activity. Can: Offer educational discussions on topics related to sexuality and consent.
Erosion of Trust in Public Information	O4	Should not: Spread misinformation or disinformation undermining public trust such as misleading medical advice, illegal drugs and advice. Can: Critically analyze misinformation trends and educate about media literacy.
Propagating Misconceptions/False Beliefs	O5	Should not: Disseminate unverified, misleading information, conspiracy theories, science denialism, historical revisionism, or content that could lead to radicalization. Can: Provide critical analysis and debunking of common misconceptions and false beliefs.
Risky Financial Practices	O6	Should not: Promote unethical trade practices such as multi-level marketing, deceptive lending practices, or gambling. Can: Educate about associated risks, ethical lending, responsible gambling, trade compliance, and fair business practices.
Trade and Compliance	O7	Should not: Involve labor issues, engage in unethical trade affairs, contribute to unfair labor practices or promote fraudulent trade practices. Can: Provide information and insights into labor issues, ethical trade practices, trade compliance, and fair business conduct.
Dissemination of Dangerous Information	O8	Should not: Spread fake news, propagate social media rumors, disclose government secrets or leak financial data and other contents that contributing to public panic or national security. Can: Educate about the dangers of such information and promote safety.
Privacy Infringement	O9	Should not: Involve doxxing, unauthorized data sharing, or surveillance like private information inferring or leaking. Can: Discuss the importance of privacy rights and digital security.
Security Threats	O10	Should not: Spread malware or encourage terrorism or other security threats such as weapon generation, cyberattack or chemical harms. Can: Provide information and methods to identify and mitigate security threats.
Defamation	O11	Should not: Publish false statements damaging to individuals, groups or organizations. Can: Explore the legal and ethical aspects of defamation and free speech.
Fraud or Deceptive Action	O12	Should not: Engage in academic dishonesty, perpetrate scams, promote pseudo-pharmaceuticals, or disseminate fake online content. Content should not deceive or defraud individuals. Can: Educate about recognizing and preventing fraudulent activities.
Influence Operations	O13	Should not: Manipulate public opinion or disrupt societal harmony for political or ideological purposes like surveillance or campaign. Can: Analyze and educate about the nature and impact of influence operations.
Illegal Activities	O14	Should not: Promote or provide guidance on engaging in illegal actions, such as crimes or terrorism. Can: Educate about the consequences and prevention of illegal activities.
Persuasion and Manipulation	O15	Should not: Involve persuasion or manipulation techniques that promote self-harm, psychological manipulation, or any form of harm to individuals' well-being. Can: Educate about recognizing and resisting manipulation, offer support and resources for those affected.
Violation of Personal Property	O16	Should not: Enagge in violating an individual's personal property,malicious mimicking, plagiarism, copyright infringement or identity forgery. Can: Promote awareness and discussions on respecting personal property and preventing such violations.

Skills that support offensive detection

Table 1. Supported skills by workflow

Workflow	Application	Supported skills
Technology	Now Assist for Configuration Management Database (CMDB)	Configuration item (CI) summarization Manage duplicate CIs Service Graph Connector diagnosis
Technology	Now Assist for ITOM	Alert analysis Alert investigation
Technology	Now Assist for IT Service Management (ITSM)	Change request risk explanation Change request summarization Chat reply recommendation Chat summarization Incident assist Incident summarization KB generation Resolution notes generation Sidebar discussion summarization
Technology	Now Assist for Security Incident Response	Post-incident analysis Resolution notes generation Security incident recommended actions Security incident summarization
Technology	Now Assist for Strategic Portfolio Management (SPM)	Multi feedback summarization Planning item doc summarization Project doc summarization Project summary emails Story generation Write planning item
Customer	Now Assist for Customer Service Management (CSM)	Case summarization Chat recommendation Chat summarization Email recommendation KB generation Resolution notes generation Sidebar summarization
Customer	Now Assist for Field Service Management (FSM)	KB generation Sidebar summarization Work order task summarization
Customer	Now Assist for Financial Services Operations (FSO)	Case summarization Disputes intake via Virtual Agent
Customer	Now Assist for Public Sector Digital Services (PSDS)	Government case summarization Chat summarization
Employee	Now Assist for Health and Safety	Incident summarization
Employee	Now Assist for HR Service Delivery (HRSD)	Case summarization Chat summarization KB generation Resolution notes generation
Employee	Now Assist for Legal Service Delivery (LSD)	Legal request summarization
Employee	Now Assist in Contract Management	Contract analysis Contract metadata extraction
Creator		Catalog item generation
Finance & Supply Chain	Now Assist for Accounts Payable Operations (APO)	Record summarization
Finance & Supply Chain	Now Assist for Supplier Lifecycle Operations (SLO)	Supplier case summarization
Finance & Supply Chain	Now Assist for Sourcing and Procurement Operations (SPO)	Record summarization

• Activate offensiveness protection for generative AI
  Activate offensiveness detection to log or block offensive content generated by Now Assist skills and workflows.
• Configure prompt injection attack protection
  Activate or deactivate prompt injection attack detection settings to protect all generative AI interactions on your instance from malicious inputs and unintended model behaviors.
• Configure sensitive topic filters
  Set up filters in Now Assist Guardian to redirect Virtual Agent users to a live agent or HR case when a sensitive subject is detected in a conversation.
• Export Now Assist Guardian logs
  Export logs from Now Assist Guardian to get insights into how often different guardrails are being detected and used.
• Configuring a Guardrail Service Provider
  Now Assist Guardian supports default guardrails and third-party or custom guardrail service providers to extend AI content monitoring with your organization's AI governance policies.