> This page is for version v2026-02-09 (default).
> For other versions, use one of these documentation indexes:
> - v2026-02-09 (default): https://docs.extend.ai/2026-02-09/llms.txt
> - v2025-04-21: https://docs.extend.ai/2025-04-21/llms.txt
> - v2024-12-23: https://docs.extend.ai/2024-12-23/llms.txt

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.extend.ai/llms.txt
> Use this file to discover all available pages before exploring further.
>
> ## API version
> The current API version is `2026-02-09`, served at the site root (no version prefix in URLs).
> If this page URL contains `/2025-04-21/` or `/2024-12-23/`, you are reading an older API version.
> Prefer the current docs at https://docs.extend.ai/llms.txt unless the user explicitly needs that older version.
> Do not treat older-version pages as the source of truth for new integrations.

# Evaluation Overview

> Evaluation sets measure the accuracy of an extractor, classifier, or splitter against reviewed ground truth. Create sets and run them via the SDK (evaluationSets, evaluationSetItems, evaluationSetRuns) or in Extend Studio.

Evaluation sets let you test the accuracy of your document processors reliably and repeatedly. An **evaluation set** is attached to a [processor](/evaluation/processors) and holds **items**: files paired with the output you expect. An **evaluation set run** scores one processor version against those items and returns accuracy metrics, so you can verify that a configuration change improved results before you publish it. Evaluation sets can also be run against a workflow to score its end-to-end output; the pages below focus on processors.

Evaluation is fully available through the API and in Extend Studio:

| Task                                             | API                                                                                                 | Studio                                                          |
| ------------------------------------------------ | --------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
| Create a set and add items with expected outputs | `POST /evaluation_sets`, then `POST /evaluation_sets/{evaluationSetId}/items`                       | Convert a reviewed batch run, or add files one at a time        |
| Run a set against a processor version            | `POST /evaluation_set_runs`, then poll `GET /evaluation_set_runs/{id}`                              | Click "Run" from the processor runner or the set's page         |
| Read results                                     | Aggregate `metrics` on `GET /evaluation_set_runs/{id}` (overall and per-field / per-class accuracy) | Metrics summary, per-document diffs, CSV export, run comparison |
| Correct ground truth                             | `POST /evaluation_sets/{evaluationSetId}/items/{itemId}`                                            | "Update" on a document's result                                 |

> **Note**
>
> Endpoint paths above are for API version `2026-02-09`. On `2025-04-21` and earlier, items are managed through `POST /evaluation_set_items`, `POST /evaluation_set_items/{id}`, and `POST /evaluation_set_items/bulk`, and evaluation set runs are started from Extend Studio rather than the API. See the API reference for the version you have pinned.

#### [Creating Evaluation Sets](/evaluation/creating-evaluation-sets)

Create a set, upload files, and add items with expected outputs, in Python, TypeScript, Java, Go, or Studio.

#### [Running Evaluation Sets](/evaluation/running-evaluation-sets)

Start a run, poll it to completion, and read accuracy metrics.

## The evaluation loop

1. **Create an evaluation set** containing examples that represent the range of documents your processor needs to handle. Aim for coverage of layouts, vendors, and edge cases rather than volume.
2. **Run it** against the processor version you are working on (the draft, or a published version).
3. **Review the results** to confirm outputs match expectations and to find common errors. Per-field accuracy tells you which parts of a schema need work.
4. **Iterate** on the processor configuration (field descriptions, rules, base processor) and rerun. Compare runs to confirm the change helped. [Composer](/optimization/composer) automates this step using your evaluation sets.
5. **Publish** the improved version once the set passes your bar, and pin it in your workflows.

## In Extend Studio

The Evaluation page in the Studio section lists your evaluation sets with options to create new sets and start runs.

![Evaluation page in Extend Studio listing evaluation sets with their processors and latest run accuracy](/_fern-img/3e02e896a8a75b2bde5ad289ae21ada1a603f3e5a603cd074876b09dd5181b7a.webp)

## Reference

* [Evaluation endpoints](/api-reference/endpoints/evaluation/create-evaluation-set): sets, items, and runs
* [Processors](/evaluation/processors): the saved, versioned entity a set is attached to
* [Calculating Array Accuracy](/evaluation/calculating-array-accuracy): how tables and lists are scored