Skip to main content
A Test simulates one conversation with your Agent or Workforce. You describe a user in the Scenario field — “You are a long-time customer who was charged twice for the same order” — and Relevance plays that user, generates their messages, and hands back the transcript. The Checks you attach score it. Tests live in a Test set, which is what a run points at — run the whole set, or a single Test inside it.

​
How to Create a Test

  1. Open your Agent or Workforce in the builder and click the Evaluate tab. Select Test from the left sidebar.
  2. Click New test set, name it in the Create new test set modal, and click Create.
  3. Click the Test set you just created to open it.
  4. Click Add test. The Add new test modal opens.
  5. Optionally pick one of the five presets under Start with a preset or write your own — Happy path, Vague request, Frustrated user, Off-topic request, or Follow-up. Each fills in the Scenario and an exact first message for you, which you can then edit.
  6. Fill in the Test:
  7. Under Checks, attach the criteria this Test is scored against. Checks pass or fail, and are combined into the Test score.
    + Add existing check opens your library of existing Checks. Pick the ones you want, up to 10 per Test.
    See Check types for what each type scores and the fields it takes.
  8. Set When the test hits an approval to control how the Test resolves Tool approvals and escalations mid-conversation:
  9. Optionally set Tool simulation. Each Tool on your Agent or Workforce has a Live / Simulated toggle. Live means the Tool actually runs during the Test — real emails sent, real records updated. Simulated replaces the call with a fake-but-realistic response. In Simulated mode you can describe what the Tool should return (leave it empty for a generic response), and open Advanced to pick the model that generates it. Simulate all tools and Reset all to live flip every Tool at once, and the section header shows how many are currently simulated. When evaluating a Workforce, Tool simulation is configured per node, so each sub-agent uses its own simulated Tool responses.
  10. Click Save test. Reopening an existing Test shows the same form titled Edit test, with Update test to save.
You can add multiple Tests to a single Test set to evaluate different aspects of its behavior. Each Test can have its own Scenario, message cap, run count, attached Checks, and Tool simulations.

​
Managing Tests

Tests can be reorganized across Test sets as your testing strategy evolves. Each Test has a dropdown menu (the three-dot icon next to the Test name):

​
Example Tests

Name: Product expertiseScenario: You are a procurement manager at a mid-sized company evaluating solutions for your team. You need specific details about enterprise pricing tiers, integration capabilities with existing tools like Salesforce and HubSpot, and data security certifications. Ask clarifying questions and compare features against competitors you’re also considering.Up to: 15 messagesCheck: Accurate information (AI Judge)
  • Evaluation Prompt: Did the Agent provide accurate product information without making claims that cannot be verified? Responses should be factual, reference actual product capabilities, and acknowledge when information needs to be confirmed by a sales representative.
Name: Escalation requestScenario: You are a paying customer who has experienced a service outage affecting your business operations. You’ve already troubleshooted with the knowledge base articles and need to speak with a senior support engineer or account manager. Be firm but professional in your request, and provide context about the business impact.Up to: 5 messagesCheck: Appropriate escalation (AI Judge)
  • Evaluation Prompt: Did the Agent acknowledge the severity of the situation, validate the customer’s need for escalation, and initiate a handoff to a human representative while maintaining a professional and empathetic tone throughout?
Next: Read the Check types reference — what each type scores and how the Checks tab works.

​
Frequently asked questions (FAQs)

You can add as many Tests as needed to a single Test set. Each Test is evaluated independently and can have its own attached Checks.
Each Test supports up to 10 Checks. This applies to Checks defined on the Test itself. Checks added under Additional checks at run time are counted separately.
Yes. Each Test has a dropdown menu (three-dot icon) with three options: Move relocates the Test to another Test set, Copy creates a duplicate in another Test set, and Duplicate creates a copy in the same Test set.