Go live and monitor
Summarize
Summary of Go live and monitor
This guide helps ServiceNow customers effectively deploy AI agents to production and establish a continuous monitoring and improvement process. It emphasizes that going live is the start of ongoing performance and security management, not the final step. Proper setup of analytics and monitoring from day one provides a performance baseline to guide future optimizations.
Show less
Deploy to Production
Before deploying, ensure all Phase 5 validation gate criteria are met and required organizational approvals, such as security reviews and AI governance sign-offs, are obtained. Verify that all critical configurations—including roles, access controls (ACLs), user identities, Guardian settings, and privacy policies—are accurately replicated from non-production to production environments, as some settings do not transfer automatically.
Activate Analytics and Monitoring
On go-live day, enable these monitoring tools to track AI agent performance and security:
- Now Assist Analytics: Monitors overall usage, performance, and adoption of Now Assist skills and AI agents.
- AI Agent Analytics dashboard: Tracks agent-specific metrics like usage, efficiency gains, time to resolution, and task closure trends.
- Now Assist Guardian analytics: Monitors guardrail effectiveness by detecting offensive content, prompt injection attempts, and sensitive topic triggers.
Establish Your Baseline
Record initial production metrics aligned with success criteria defined earlier to establish a reference point for future reviews. Key metrics to capture from day one include:
- Agent execution volume (number of invocations)
- Task closure and deflection rates
- Guardian detection counts by category (offensive content, prompt injections, sensitive topics)
- Access control failures found in logs
Ongoing Monitoring and Review
Maintain agent performance and security by scheduling regular reviews:
- Guardian log review: Export and analyze Guardian logs routinely to detect changes in offensive or suspicious content patterns, which might indicate targeting or the need for prompt adjustments.
- Access control audit: Periodically validate role assignments and role masking to ensure least-privilege access aligns with evolving organizational structures.
- Performance review: Compare ongoing performance data against baseline and success criteria using analytics dashboards to identify trends and decide on prompt refinements or configuration updates.
Deploy your agent to production, activate analytics and monitoring, and establish the ongoing review cadence that keeps your agent performing securely over time.
Deploying to production is not the end of the process — it is the beginning of an ongoing monitoring and improvement cycle. Set up your monitoring before you go live so that you have a performance baseline from the first day of production operation.
Deploy to production
Before deploying, confirm that all Phase 5 validation gate criteria have been met and that any required organizational approvals (security review, AI governance sign-off) have been obtained.
When deploying, verify that all roles, ACLs, user identity configurations, Guardian settings, and privacy policies from your non-production build are correctly replicated in your production instance. Access control and Guardian configuration does not transfer automatically in all deployment scenarios.
Activate analytics and monitoring
Set up the following monitoring capabilities on the day of go-live:
- Now Assist Analytics
- Monitor overall usage, performance, and adoption of your Now Assist skills and AI agents. See Analyzing Now Assist performance.
- AI Agent Analytics dashboard
- Track AI agent use, efficiency gain, time to resolution, and task closure trends specific to your deployed agents. See AI Agent Analytics dashboard.
- Now Assist Guardian analytics
- Monitor guardrail performance to track how often offensive content, prompt injection attempts, and sensitive subject detections occur in production. See Now Assist Guardian analytics.
Establish your baseline
Record your initial production metrics against the success criteria you defined in Phase 2. Your baseline measurements during the first week of production operation give you the reference point for all future optimization decisions.
At minimum, record the following from day one:
- Agent execution volume (number of invocations).
- Task closure and deflection rate.
- Guardian detection counts by category (offensive, prompt injection, sensitive topics).
- Any access control failures surfaced in logs.
Ongoing monitoring and review
Schedule the following recurring activities to keep your agent performing securely:
- Regular Guardian log review
- Export and review Now Assist Guardian logs on a regular schedule to identify emerging patterns in detected content. See . Increasing detection rates may indicate that your agent is being targeted or that your prompts need refinement.
- Access control audit
- Periodically review who holds the roles that can invoke your agents, and confirm that role masking configurations still reflect least-privilege intent as your organization's roles and team structure evolve.
- Performance review
- Review your agent's performance metrics against your baseline and success criteria. Use the AI Agent Analytics dashboard and Now Assist Analytics to identify trends and determine whether prompt refinement or configuration changes are needed.