Create Evaluation Set | Extend Documentation

from extend_ai import Extend
client = Extend(
    token="YOUR_TOKEN_HERE",
    extend_api_version="2026-02-09",
)
client.evaluation_sets.create(
    name="Invoice Processing Test Set",
    description="Q4 vendor invoices for accuracy testing",
    entity_id="ex_1234567890",
)

{
  "object": "evaluation_set",
  "id": "ev_2LcgeY_mp2T5yPaEuq5Lw",
  "name": "Invoice Processing Test Set",
  "description": "Q4 2023 vendor invoices for accuracy testing",
  "entity": {
    "object": "extractor",
    "id": "ex_Xj8mK2pL9nR4vT7qY5wZ",
    "name": "Invoice Extractor",
    "createdAt": "2024-03-21T16:45:00Z",
    "updatedAt": "2024-03-21T16:45:00Z"
  },
  "createdAt": "2024-03-21T16:45:00Z",
  "updatedAt": "2024-03-21T16:45:00Z",
  "createdBy": {
    "type": "user",
    "email": "matt@extend.app"
  }
}

Evaluation sets are collections of files and expected outputs that are used to evaluate the performance of a given extractor, classifier, or splitter. This endpoint will create a new evaluation set, which items can be added to using the Create Evaluation Set Item endpoint.

Note: It is not necessary to create an evaluation set via API. You can also create an evaluation set via the Extend dashboard and take the ID from there. To learn more about how to create evaluation sets, see the Evaluation Sets product page.

Authentication

AuthorizationBearer

Bearer authentication of the form Bearer <token>, where token is your auth token.

Request

This endpoint expects an object.

namestringRequired

The name of the evaluation set.

Example: "Invoice Processing Test Set"

entityIdstringRequired

The ID of the extractor, classifier, or splitter to create an evaluation set for. Evaluation sets can in theory be run against any extractor, classifier, or splitter, but it is required to associate the evaluation set with a primary extractor, classifier, or splitter.

Example: "ex_Xj8mK2pL9nR4vT7qY5wZ"

descriptionstringOptional

A description of what this evaluation set is used for.

Example: "Q4 2023 vendor invoices"

Response

Successfully created evaluation set

objectenum

The type of object. In this case, it will always be "evaluation_set".

Allowed values:

idstring

The ID of the evaluation set.

Example: "ev_2LcgeY_mp2T5yPaEuq5Lw"

namestring

The name of the evaluation set.

Example: "Invoice Processing Test Set"

descriptionstring

A description of the evaluation set.

Example: "Q4 2023 vendor invoices for accuracy testing"

entityobject

The extractor, classifier, or splitter associated with this evaluation set.

createdAtstringformat: "date-time"

The time (in UTC) at which the object was created. Will follow the RFC 3339 format.

Example: "2024-03-21T16:45:00Z"

updatedAtstringformat: "date-time"

The time (in UTC) at which the object was last updated. Will follow the RFC 3339 format.

Example: "2024-03-21T16:45:00Z"

createdByobject

The actor that created this resource.

user — the resource was created via the studio by a logged-in user. The user’s email is exposed.
api_key — the resource was created via the API using an API key. The key’s name is exposed.

May be null for resources created before attribution.

Errors

400

Bad Request Error

401

Unauthorized Error

402

Payment Required Error

403

Forbidden Error

404

Not Found Error

422

Unprocessable Entity Error

429

Too Many Requests Error

500

Internal Server Error

Authentication

Headers

Request

Response

Errors