Design considerations for prompting
Summarize
Summarized using AI
This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.
Summary of Design considerations for prompting
Starting with the Australia release, Test generation is a Now Assist backed feature designed to create Automated Test Framework (ATF) tests efficiently. However, this feature is being prepared for future deprecation; it will no longer be installed on new instances but will continue to be supported on existing ones. Test generation specifically focuses on creating ATF tests within the current application scope and does not cover other types of functional or integration tests.
Show less
Key Capabilities and Scope
- Creates ATF tests only within the active application scope.
- Does not support updating or deleting existing ATF tests.
- Supports a range of test scenarios including:
- Form interactions like open/submit, field validation/visibility, UI/Declarative Action, Modal buttons.
- Supports workspace form UI types such as serviceoperationsworkspace, assetworkspace, and cmdbworkspace.
- Server operations including impersonation, user creation, logging, record CRUD, replaying requests, catalog item search, and shopping cart checkout.
- Email operations: generating, replying, sending, and validating inbound/outbound emails.
- Application Navigator tasks: module navigation and menu/module visibility.
- Reporting: visibility of reports and dashboards, including sharing.
- Service Catalog operations: opening/ordering catalog items, adding to shopping carts, setting/validating item quantities, pricing, and variable status.
Effective Prompting Principles
Clear, concise, and well-defined prompts are essential to generate accurate and useful ATF tests. Effective prompts should:
- Precisely define user roles and actions, such as creating or impersonating a user.
- Clearly specify field and record details, including exact field names, values, and expected outcomes.
- Include necessary keywords like "create," "update," "set," or "validate" to clarify the intended actions.
- Explicitly name catalog items and include all relevant attributes such as color, data allowances, and pricing with correct units and currency symbols.
In contrast, vague or incomplete prompts lead to ambiguous test generation, causing unclear or incorrect test steps.
Practical Implications for ServiceNow Customers
- When using Test generation, customers should carefully craft prompts that fully describe the desired test steps to ensure accurate ATF test creation.
- Since Test generation will no longer be installed on new instances, customers should plan for alternative test creation strategies in the future.
- Understanding the supported form types and operations helps customers design tests within the tool’s current capabilities.
Generate your desired test by following the guided principles of effective prompting.
Starting with the Australia release, Test generation is being prepared for future deprecation. It will be hidden and no longer installed on new instances but will continue to be supported. For details, see the Deprecation Process [KB0867184] article in the Now Support Knowledge Base.
- Clear and concise ATF prompts that describes test steps
- Consider the scope and capability of Test generation
- Test generation is a Now Assist backed skill to create ATF tests
- The tests are created in the application scope that you are currently on
- Other types of functional or integration tests are out of Test generation scope
- Test generation can't update or delete existing ATF tests
- Test generation supports the following
- Forms: open/submit form, field validation/visibility, UI/Declarative Action and Modal button. Each form API supports workspace as formUI type, including service_operations_workspace and asset_workspace and cmdb_workspace.
- Server operations: impersonation/create user, log message, record CRUD, replay request, search catalog item and check out shopping cart.
- Email: generate, reply, send and validate inbound / outbound email
- Application Navigator: navigate to module, menu/module visibility
- Reporting: report visibility, dash board visibility and sharing
- Service Catalog: open/order catalog item, add item to shopping cart, set/validate item quantity, submit order, validate quantity, price, and variable status
| Good and effective prompts | Bad and irrelevant prompts | Comparison description |
|---|---|---|
| Write a test to create a new user with name Bill McDermott and assign the "itil" role. Create a new incident record, update the short description to "my new test record" and then validate the record description. Update caller to Bill McDermott, impact and urgency categories to "2- Medium" and submit the record. Delete the record and create a log for the deletion. | I need a new user Bill McDermott to open a new record. Make sure its called my new test record. Update caller to user, impact and urgency categories to Med. Delete the record and create a log for the deletion. | The bad prompt is irrelevant and incomplete for the following reasons:
|
| Write an ATF test, named 'incident record test', impersonate admin, create a new incident record, then update the record short description to 'test update field from example', delete the record, then log a message 'record deleted' | Create an ATF test for an admin to create an incident record. Make description test update field from example. Then delete the record | The bad prompt lacks the following:
|
| Write a test to open Apple iPhone 13 catalog item, set color to 'pink', monthly data allowance to 500MB, validate price to be $800.00 and log the price | Write a test to open iPhone, color should be pink and data should be 500. The price should be 800. Log the price | The bad prompt is incomplete for the following reasons:
|