Evaluate a prompt

  • Release version: Australia
  • Updated March 12, 2026
  • 2 minutes to read

    • Use the Now Assist Skill Kit evaluation tools to evaluate the effectiveness of your skill prompts.

    Before you begin

    Role required: sn_skill_builder.admin

    Procedure

    1. Navigate to All > Now Assist Skill Kit > Home.
    2. Select the skill that you want to evaluate.
    3. Select the Prompt performance tab.
    4. Select the Evaluation runs tab.
    5. Create a dataset from a table or data collection.
      Table 1. Create a data set
      Method Steps
      Create a dataset from a table
      1. Give the dataset a name and description.
      2. Select Table.
      3. Find the table that you want to use.
      4. Select the maximum number of records that you want to use.
      5. Add conditions.
      6. Select Generate Preview.
      7. Select the mappings.
      8. Select Create.
      Create a dataset from a data collection
      1. Give the dataset a name and description.
      2. Select Data Collection.
      3. Select a data collection that you created in Now Assist Data Kit.
      4. Select Generate Preview.
      5. Select the mappings.
      6. Select Create.
    6. Select the add icon add icon for Evaluation Runs.
    7. Give the evaluation run a name and description.
    8. Select one or more prompts that you want to evaluate.
    9. Select Save & Next.
    10. Select a dataset.
    11. Select Save & Next.
    12. Expand the Quality tab.
    13. Select the metrics that you want to evaluate.
      Table 2. Evaluation metrics
      Evaluation method Metric Description
      Human Human Feedback Human evaluation is the default option available for all prompt executions that generate a response. You can rate the response with a thumbs up or thumbs down, based on your satisfaction. You also have the option to provide more detailed feedback to explain your evaluation choice.
      Automated Correctness The correctness metric assesses the generated response's accuracy, completeness, pertinence, and writing quality relative to the given instruction. This metric helps to check that the text accurately reflects the instruction, covers all important points, remains relevant, and is well written.
      Automated Correctness with Golden Response The correctness with golden response metric uses a predefined reference to assess the generated response's accuracy, completeness, pertinence, and writing quality relative to the given instruction. This metric helps to check that the text accurately reflects the instruction, covers all important points, remains relevant, and is well written. You should use this metric whenever possible.
      Automated Faithfulness The faithfulness metric assesses whether a generated response accurately reflects the information and context provided in the given instruction. This metric helps to check that the text contains no hallucinations, fabricated facts, or unsupported conclusions, maintaining alignment with the source material.
    14. Select Save & Next.
    15. Review the evaluation choices that you made.
    16. Select Save & Evaluate.
    17. Optional: Give a human evaluation.
      1. Select Human evaluation.
      2. Select a record to use in the evaluation.
      3. Expand the prompt and read the result.
      4. Select the thumbs up or thumbs down icon human evaluation thumbs up or thumbs down icon to give your evaluation.
      5. Add more information and select Submit.