Skip to main content
A Check is one pass-or-fail criterion. Every Agent and Workforce has its own library of them in the Checks tab, and the same Check can be attached in three places:
  • To a Test in a Test set — it runs every time that Test is evaluated.
  • To a Monitor dashboard — it runs on a sampled portion of live tasks.
  • To a one-off evaluation of already-completed tasks selected from the task list.
This page is the reference for what each type scores. To attach one while building a Test, see creating Tests.

​
Check types

When creating a Check, you choose one of the following types:
Uses AI to evaluate the response against criteria you define.
Checks whether the response includes specific text.
Checks whether the response exactly matches an expected value.
Checks whether a specific Tool was used during the conversation.You can also assert what the tool was called with. Click Add input condition to check that an input matched — Equals, Contains, Has a value, or Matches pattern for a regular expression.When evaluating a Workforce, you can scope a Tool Usage Check to a specific node — a sub-agent or tool in the Workforce — so you can assert that a particular sub-agent used a given tool.

​
Creating a Check in the Checks tab

Checks can be created from inside a Test as you build it, or on their own here in the library. Either route produces the same reusable Check.
  1. Go to the Evaluate tab and select Checks from the left sidebar.
  2. Click New check.
  3. Enter a Check name (e.g., “Professional tone”), then choose a type from the Select check type… picker.
  4. Fill in the fields for that type — see Check types above.
  5. Click Save check. Reopening an existing Check shows Update check instead.
Each Test supports up to 10 Checks of its own. Checks attached to a Test are always included when you run that Test. Additional Checks from the Checks tab are not auto-included — expand Additional options in the run modal and select them under Additional checks before kicking off the run. Those are counted separately from the per-Test limit.

​
The Checks tab

The Checks tab lists every Check on the Agent or Workforce, grouped by type. Filters across the top show where each one is currently attached:
Next: Learn how to run Evals against your Test sets.

​
Frequently asked questions (FAQs)

No. Creating one with + New check while building a Test saves it to your library like any other, so you can attach it to other Tests and to Monitor dashboards afterwards.
Each Test supports up to 10 Checks of its own. Checks added under Additional checks at run time are counted separately.
Yes, with configuration. The AI Judge Check includes a Truncate long conversations toggle in the Advanced section when creating a Check. When enabled, conversations that exceed the judge model’s context window are trimmed and evaluated. When disabled, those conversations fail with an error rather than producing a partial result.
The oldest messages are removed from the start of the conversation until it fits within the judge model’s context window. The judge is notified that truncation occurred and evaluates the remaining portion. If your evaluation criteria depend on early context — such as the user’s original request or instructions given at the start of the conversation — the result may be less accurate. In those cases, disabling truncation and selecting a model with a larger context window is preferable.