> ## Documentation Index
> Fetch the complete documentation index at: https://docs.traceport.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluations

> Define scoring rules to automatically grade prompt quality, relevance, and safety.

**Evaluations** provide an automated testing framework for your prompts. Define scoring criteria and let Traceport grade every prompt output against your quality standards.

## What Are Evaluations?

Evaluations are rules that automatically score the output of your prompts. Instead of manually reviewing every response, define criteria once and Traceport evaluates every test run.

## Creating Evaluation Rules

<Steps>
  <Step title="Open Evaluations">
    Click the **Evaluations** icon in the Prompt Studio sidebar.
  </Step>

  <Step title="Add Rule">
    Click **Add Evaluation Rule** and define a scoring criterion.
  </Step>

  <Step title="Configure">
    Set the evaluation type, scoring method, and pass/fail threshold.
  </Step>

  <Step title="Run">
    Execute your prompt — evaluation scores appear alongside the output.
  </Step>
</Steps>

## Evaluation Types

<AccordionGroup>
  <Accordion title="Content Quality">
    Score the response's relevance, coherence, and completeness relative to the user's request. Catches off‑topic or low‑quality responses.
  </Accordion>

  <Accordion title="Safety & Compliance">
    Check for harmful content, PII leakage, or policy violations. Essential for customer-facing applications.
  </Accordion>

  <Accordion title="Format Validation">
    Verify that the response follows a required format — JSON schema, specific structure, or required fields.
  </Accordion>

  <Accordion title="Custom Criteria">
    Define your own scoring logic using natural language descriptions. Traceport uses an evaluator model to grade responses against your criteria.
  </Accordion>
</AccordionGroup>

## Evaluations + Datasets

The most powerful workflow combines Evaluations with [Datasets](/prompt-studio/testing):

1. **Create a Dataset** with diverse test inputs
2. **Define Evaluation Rules** for quality, safety, and format
3. **Run the batch** — Traceport evaluates every response against every rule
4. **Review the scorecard** — identify which inputs produce failing outputs

<Info>
  This workflow is ideal for prompt optimization cycles: make a change, run the dataset, and compare evaluation scores before and after.
</Info>

## Continuous Evaluation

As your prompts evolve through new versions, evaluations serve as a quality gate:

* Run evaluations before publishing a new version
* Compare scores across versions to detect regressions
* Use evaluation pass rates as confidence signals for deployment

<Warning>
  Evaluations use an additional model call to score responses. This adds a small cost per evaluation. Use them strategically on important prompts rather than on every prompt run.
</Warning>