Now Assist Data Kit
Every week, practitioners across the ServiceNow ecosystem are quietly solving one of AI’s most overlooked problems, not the model, not the prompt, but the data. This series is about their stories. I’m Ragu Ramakrishnan, Product Manager for Now Assist Data Kit and I started the AI Data Mastery Series to bring those stories to the surface real people, real problems and real solutions.
The first story belongs to Yogesh Shinde — because before you can trust your AI, you have to trust your data. And Yogesh learned that the hard way.
@Yogesh Shinde is a ServiceNow AI Architect, who helps enterprises move from AI experiments to production grade agents designing, shipping and governing agentic solutions across the full stack, from Now Assist skills to multi‑agent workflows and cross‑platform integrations.
The Problem Nobody Warns You About
When I began exploring Now Assist skills and agentic workflows in sub-production instances, the blocker wasn’t the model or the prompt — it was the data. The environments were sparse, and I couldn’t use real customer production records due to governance constraints. Manual record creation didn’t scale, and it biased the test set toward “clean” scenarios. The result was misleading: retrieval and summarization quality looked inconsistent because the inputs weren’t representative. I needed a repeatable way to create realistic incidents, cases, and knowledge test scenarios at volume — so I could evaluate whether behavior changes were caused by configuration or prompt changes, or simply by poor data. That’s when it became obvious: if the dataset isn’t credible, the AI assessment isn’t credible.
Why This Problem Is Harder Than It Looks
Here’s the reality that Yogesh’s story puts a spotlight on.
When a Solution Consultant or AI Architect spins up a sub-production instance to explore Now Assist skills or agents, they face three walls simultaneously. The instance is sparse — few records, little variety. Real customer data is off limits, as privacy, compliance, and governance all rightly prevent it from moving into demo or POC environments. And building records manually doesn’t scale — a half-day of hand-crafting incidents to make one demo look real isn’t a workflow, it’s a tax.
The result is a strange situation: genuinely powerful AI capabilities running against data that makes them look mediocre. The model isn’t the problem. The data is.
This is exactly the gap Now Assist Data Kit’s Synthetic Data Generator was built to close.
What the Synthetic Data Generator Actually Does
Now Assist Data Kit gives you two generation modes depending on your use case.
The Standard Generator creates flat records for a single table such as incidents, cases, HR cases, knowledge articles, based on a scenario you describe. You tell it the industry, the department, and the kinds of issues the data should reflect. The more specific you are, the more realistic the output. You can also seed it with a few example records to anchor the language and structure, which makes a significant difference in output quality.
The Multi-table Generator is for more complex scenarios, when your agent or skill needs to traverse related records across multiple tables. It generates data across all selected tables in a single run, maintaining referential integrity automatically. So an HR case that should relate to a user record and trigger a task actually does, rather than existing in isolation.
Both modes give you a preview batch before you commit to full-scale generation. It’s worth taking five minutes to review and refine — it almost always saves time downstream.
To get started: All > Now Assist Data Kit > Home > Synthetic datasets tab > Generate dataset.
Finding Data Kit — Yogesh’s Turning Point
I moved to Now Assist Data Kit because I needed a structured, scalable way to generate test data and evaluation assets, rather than relying on manually hand-crafted records. As I worked through creating datasets, synthetic datasets, derived datasets and publishing a collection, it became clear how powerful the capability could be. One thing I learned early: generation time is directly tied to LLM calls and increases with longer column types — such as JSON or large text fields. Knowing that upfront helps you design your datasets more intentionally from the start.
How Yogesh Uses It Today
My approach is intent-first. I start by defining what I want to test, prompt changes, summary quality, evaluation repeatability and then design the dataset to support that goal. In practice, I create a base dataset, generate or add synthetic data and then derive a smaller, targeted subset when needed. I’ve worked primarily with single-table datasets so far.
Because synthetic data generation relies on multiple LLM calls per record, runtime varies based on dataset size and schema complexity. Larger schemas, especially those with long text fields or JSON columns, result in slower generation. To manage this, I limit generation to only the fields required for the test scenario, generate data in smaller batches, and prefer creating derived datasets rather than regenerating large datasets repeatedly.
For evaluation readiness, I treat ground truth as a deliberate, separate step via labelling projects. Once that’s done, I publish a data collection for consumption in Now Assist Skill Kit evaluation. The key is being intentional at each stage rather than rushing to publish.
What It Changed
The workflow is conceptually sound and once you understand the end-to-end flow from synthetic data through to Skill Kit evaluation, it becomes a genuinely powerful part of how you approach AI quality work.
From Exploration to Evaluation
One thing worth calling out: the data Yogesh builds for exploration doesn’t have to be throwaway.
A well-constructed synthetic dataset grounded in a realistic scenario and seeded with good examples is also a strong starting point for a formal evaluation dataset. In Now Assist Data Kit, that same dataset can be curated into a data collection, published and fed into Now Assist Skill Kit’s auto-evaluation to measure how an OOB or customised skill performs before go-live. The work compounds.
Yogesh’s Advice to Anyone Starting Out
Don’t evaluate Now Assist on sparse or hand-crafted toy data. Use Now Assist Data Kit to generate datasets and establish a repeatable baseline. Then curate a smaller derived dataset and add ground truth intentionally via labelling projects. Publish collections only when you’re ready to use them in Now Assist Skill Kit evaluation. And plan for iteration the end-to-end setup rewards patience. Budget time for it and the payoff is a far more credible evaluation process.
Try It Yourself
Stop manually writing records. Your instance already has everything you need.
All > Now Assist Data Kit > Home > Synthetic datasets > Generate dataset. Describe your scenario specifically. Seed with 2–3 example records if you have them. Run a preview. Iterate once. Generate.
Twenty to thirty minutes for a dataset that used to take half a day and one that actually makes Now Assist look like what it is.
Could this be your story?
Every article in this series is built around a real practitioner and a real problem they solved. If you’re using Now Assist Data Kit for exploring, experimenting, building, then I want to hear from you. Tell me your story and you could be the champion voice in our next article.
Reach out to Ragu Ramakrishnan on the ServiceNow Community, let’s tell your story next.
Yogesh Shinde is an AI Architect, Product Excellence at ServiceNow.
Ragu Ramakrishnan is a Product Manager, Now Assist Data Kit at ServiceNow.
