> This page is for version v2026-02-09 (default).
> For other versions, use one of these documentation indexes:
> - v2026-02-09 (default): https://docs.extend.ai/2026-02-09/llms.txt
> - v2025-04-21: https://docs.extend.ai/2025-04-21/llms.txt
> - v2024-12-23: https://docs.extend.ai/2024-12-23/llms.txt

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.extend.ai/llms.txt
> Use this file to discover all available pages before exploring further.
>
> ## API version
> The current API version is `2026-02-09`, served at the site root (no version prefix in URLs).
> If this page URL contains `/2025-04-21/` or `/2024-12-23/`, you are reading an older API version.
> Prefer the current docs at https://docs.extend.ai/llms.txt unless the user explicitly needs that older version.
> Do not treat older-version pages as the source of truth for new integrations.

# Create Evaluation Set

POST https://api.extend.ai/evaluation_sets
Content-Type: application/json

Evaluation sets are collections of files and expected outputs that are used to evaluate the performance of a given extractor, classifier, or splitter. This endpoint will create a new evaluation set, which items can be added to using the [Create Evaluation Set Item](https://docs.extend.ai/2026-02-09/api-reference/endpoints/evaluation/create-evaluation-set-item) endpoint.

Note: It is not necessary to create an evaluation set via API. You can also create an evaluation set via the Extend dashboard and take the ID from there. To learn more about how to create evaluation sets, see the [Evaluation Sets](https://docs.extend.ai/2026-02-09/evaluation/overview) product page.


Reference: https://docs.extend.ai/api-reference/endpoints/evaluation/create-evaluation-set

## Authentication

- `Authorization` header (bearer token, required) — Bearer authentication of the form `Bearer <token>`, where token is your auth token.

## Servers

- `https://api.extend.ai` (Production, default)
- `https://api.us2.extend.app` (Production US2)
- `https://api.eu1.extend.ai` (Production EU1)

## Request

### Headers

- `x-extend-api-version` ("2026-02-09", optional, default: 2026-02-09) — API version to use for the request. If you're using an SDK, you can ignore this parameter. If you are not using an SDK and do not specify a version, you will either receive a `400 Bad Request` or be set to a previous legacy version. See [API Versioning](https://docs.extend.ai/2026-02-09/api-reference/api-versioning) for more details.

### Body (application/json)

This endpoint expects an object.

- `name` (string, required) — The name of the evaluation set. Example: `"Invoice Processing Test Set"`
- `entityId` (string, required) — The ID of the extractor, classifier, or splitter to create an evaluation set for. Evaluation sets can in theory be run against any extractor, classifier, or splitter, but it is required to associate the evaluation set with a primary extractor, classifier, or splitter. Example: `"ex_Xj8mK2pL9nR4vT7qY5wZ"`
- `description` (string, optional) — A description of what this evaluation set is used for. Example: `"Q4 2023 vendor invoices"`

## Response

### 200

Successfully created evaluation set

- `object` (enum, required) — The type of object. In this case, it will always be `"evaluation_set"`.
  - Allowed values: `evaluation_set`
- `id` (string, required) — The ID of the evaluation set. Example: `"ev_2LcgeY_mp2T5yPaEuq5Lw"`
- `name` (string, required) — The name of the evaluation set. Example: `"Invoice Processing Test Set"`
- `description` (string, required) — A description of the evaluation set. Example: `"Q4 2023 vendor invoices for accuracy testing"`
- `entity` (EvaluationSetEntity, required) — The extractor, classifier, or splitter associated with this evaluation set.
- `createdAt` (string, required) — The time (in UTC) at which the object was created. Will follow the RFC 3339 format. Example: `"2024-03-21T16:45:00Z"`
- `updatedAt` (string, required) — The time (in UTC) at which the object was last updated. Will follow the RFC 3339 format. Example: `"2024-03-21T16:45:00Z"`
- `createdBy` (CreatedBy, required) — The actor that created this resource. * `user` — the resource was created via the studio by a logged-in user. The user's `email` is exposed. * `api_key` — the resource was created via the API using an API key. The key's `name` is exposed. May be `null` for resources created before attribution.

## Errors

### 400 Bad Request Error

Standard error response format for all Extend API errors. See the [Error Codes documentation](https://docs.extend.ai/2026-02-09/api-reference/error-handling) for error handling recommendations.

- `code` (string, required) — Error code for programmatic handling.
- `message` (string, required) — Human-readable error message suitable for displaying to developers. Do not rely on the exact text of this message as it may change.
- `retryable` (boolean, required) — Whether the request can be retried. When true, retry with exponential backoff. When false, fix the underlying issue before retrying.
- `requestId` (string, optional) — Unique request identifier for support purposes. Always include this when contacting Extend support about an error.
- `docUrl` (string, optional) — Link to relevant documentation when one is available.

### 401 Unauthorized Error

Standard error response format for all Extend API errors. See the [Error Codes documentation](https://docs.extend.ai/2026-02-09/api-reference/error-handling) for error handling recommendations.

- `code` (string, required) — Error code for programmatic handling.
- `message` (string, required) — Human-readable error message suitable for displaying to developers. Do not rely on the exact text of this message as it may change.
- `retryable` (boolean, required) — Whether the request can be retried. When true, retry with exponential backoff. When false, fix the underlying issue before retrying.
- `requestId` (string, optional) — Unique request identifier for support purposes. Always include this when contacting Extend support about an error.
- `docUrl` (string, optional) — Link to relevant documentation when one is available.

### 402 Payment Required Error

Standard error response format for all Extend API errors. See the [Error Codes documentation](https://docs.extend.ai/2026-02-09/api-reference/error-handling) for error handling recommendations.

- `code` (string, required) — Error code for programmatic handling.
- `message` (string, required) — Human-readable error message suitable for displaying to developers. Do not rely on the exact text of this message as it may change.
- `retryable` (boolean, required) — Whether the request can be retried. When true, retry with exponential backoff. When false, fix the underlying issue before retrying.
- `requestId` (string, optional) — Unique request identifier for support purposes. Always include this when contacting Extend support about an error.
- `docUrl` (string, optional) — Link to relevant documentation when one is available.

### 403 Forbidden Error

Standard error response format for all Extend API errors. See the [Error Codes documentation](https://docs.extend.ai/2026-02-09/api-reference/error-handling) for error handling recommendations.

- `code` (string, required) — Error code for programmatic handling.
- `message` (string, required) — Human-readable error message suitable for displaying to developers. Do not rely on the exact text of this message as it may change.
- `retryable` (boolean, required) — Whether the request can be retried. When true, retry with exponential backoff. When false, fix the underlying issue before retrying.
- `requestId` (string, optional) — Unique request identifier for support purposes. Always include this when contacting Extend support about an error.
- `docUrl` (string, optional) — Link to relevant documentation when one is available.

### 404 Not Found Error

Standard error response format for all Extend API errors. See the [Error Codes documentation](https://docs.extend.ai/2026-02-09/api-reference/error-handling) for error handling recommendations.

- `code` (string, required) — Error code for programmatic handling.
- `message` (string, required) — Human-readable error message suitable for displaying to developers. Do not rely on the exact text of this message as it may change.
- `retryable` (boolean, required) — Whether the request can be retried. When true, retry with exponential backoff. When false, fix the underlying issue before retrying.
- `requestId` (string, optional) — Unique request identifier for support purposes. Always include this when contacting Extend support about an error.
- `docUrl` (string, optional) — Link to relevant documentation when one is available.

### 422 Unprocessable Entity Error

Standard error response format for all Extend API errors. See the [Error Codes documentation](https://docs.extend.ai/2026-02-09/api-reference/error-handling) for error handling recommendations.

- `code` (string, required) — Error code for programmatic handling.
- `message` (string, required) — Human-readable error message suitable for displaying to developers. Do not rely on the exact text of this message as it may change.
- `retryable` (boolean, required) — Whether the request can be retried. When true, retry with exponential backoff. When false, fix the underlying issue before retrying.
- `requestId` (string, optional) — Unique request identifier for support purposes. Always include this when contacting Extend support about an error.
- `docUrl` (string, optional) — Link to relevant documentation when one is available.

### 429 Too Many Requests Error

Standard error response format for all Extend API errors. See the [Error Codes documentation](https://docs.extend.ai/2026-02-09/api-reference/error-handling) for error handling recommendations.

- `code` (string, required) — Error code for programmatic handling.
- `message` (string, required) — Human-readable error message suitable for displaying to developers. Do not rely on the exact text of this message as it may change.
- `retryable` (boolean, required) — Whether the request can be retried. When true, retry with exponential backoff. When false, fix the underlying issue before retrying.
- `requestId` (string, optional) — Unique request identifier for support purposes. Always include this when contacting Extend support about an error.
- `docUrl` (string, optional) — Link to relevant documentation when one is available.

### 500 Internal Server Error

Standard error response format for all Extend API errors. See the [Error Codes documentation](https://docs.extend.ai/2026-02-09/api-reference/error-handling) for error handling recommendations.

- `code` (string, required) — Error code for programmatic handling.
- `message` (string, required) — Human-readable error message suitable for displaying to developers. Do not rely on the exact text of this message as it may change.
- `retryable` (boolean, required) — Whether the request can be retried. When true, retry with exponential backoff. When false, fix the underlying issue before retrying.
- `requestId` (string, optional) — Unique request identifier for support purposes. Always include this when contacting Extend support about an error.
- `docUrl` (string, optional) — Link to relevant documentation when one is available.

## Types

### EvaluationSetEntity

The extractor, classifier, or splitter associated with this evaluation set.

### CreatedBy

The actor that created this resource. * `user` — the resource was created via the studio by a logged-in user. The user's `email` is exposed. * `api_key` — the resource was created via the API using an API key. The key's `name` is exposed. May be `null` for resources created before attribution.

- `type`: `user` (user)
  - `email` (string, required) — The email address of the user who created the resource.
- `type`: `api_key` (api_key)
  - `name` (string, required) — The name of the API key that created the resource.

### ExtractorSummary

- `object` (enum, required) — The type of object. Will always be `"extractor"`.
  - Allowed values: `extractor`
- `id` (string, required) — The ID of the extractor. Example: `"ex_Xj8mK2pL9nR4vT7qY5wZ"`
- `name` (string, required) — The name of the extractor. Example: `"Invoice Extractor"`
- `createdAt` (string, required) — The time (in UTC) at which the object was created. Will follow the RFC 3339 format. Example: `"2024-03-21T16:45:00Z"`
- `updatedAt` (string, required) — The time (in UTC) at which the object was last updated. Will follow the RFC 3339 format. Example: `"2024-03-21T16:45:00Z"`

### ClassifierSummary

- `object` (enum, required) — The type of object. Will always be `"classifier"`.
  - Allowed values: `classifier`
- `id` (string, required) — The ID of the classifier. Example: `"cl_Xj8mK2pL9nR4vT7qY5wZ"`
- `name` (string, required) — The name of the classifier. Example: `"Document Type Classifier"`
- `createdAt` (string, required) — The time (in UTC) at which the object was created. Will follow the RFC 3339 format. Example: `"2024-03-21T16:45:00Z"`
- `updatedAt` (string, required) — The time (in UTC) at which the object was last updated. Will follow the RFC 3339 format. Example: `"2024-03-21T16:45:00Z"`

### SplitterSummary

- `object` (enum, required) — The type of object. Will always be `"splitter"`.
  - Allowed values: `splitter`
- `id` (string, required) — The ID of the splitter. Example: `"spl_Xj8mK2pL9nR4vT7qY5wZ"`
- `name` (string, required) — The name of the splitter. Example: `"Invoice Packet Splitter"`
- `createdAt` (string, required) — The time (in UTC) at which the object was created. Will follow the RFC 3339 format. Example: `"2024-03-21T16:45:00Z"`
- `updatedAt` (string, required) — The time (in UTC) at which the object was last updated. Will follow the RFC 3339 format. Example: `"2024-03-21T16:45:00Z"`

## Examples

**Request**

```json
{
  "name": "Invoice Processing Test Set",
  "entityId": "ex_1234567890",
  "description": "Q4 vendor invoices for accuracy testing"
}
```

**Response**

```json
{
  "object": "evaluation_set",
  "id": "ev_2LcgeY_mp2T5yPaEuq5Lw",
  "name": "Invoice Processing Test Set",
  "description": "Q4 2023 vendor invoices for accuracy testing",
  "entity": {
    "object": "extractor",
    "id": "ex_Xj8mK2pL9nR4vT7qY5wZ",
    "name": "Invoice Extractor",
    "createdAt": "2024-03-21T16:45:00Z",
    "updatedAt": "2024-03-21T16:45:00Z"
  },
  "createdAt": "2024-03-21T16:45:00Z",
  "updatedAt": "2024-03-21T16:45:00Z",
  "createdBy": {
    "type": "user",
    "email": "matt@extend.app"
  }
}
```