Create Evaluation Set Items

POST

/evaluation_sets/:evaluationSetId/items

POST

/evaluation_sets/:evaluationSetId/items

from extend_ai import Extend
client = Extend(
    token="YOUR_TOKEN_HERE",
    extend_api_version="2026-02-09",
)
client.evaluation_set_items.create(
    evaluation_set_id="evaluation_set_id_here",
    items=[
        {
            "file_id": "string"
        }
    ],
)

{
  "evaluationSetItems": [
    {
      "object": "evaluation_set_item",
      "id": "evi_kR9mNP12Qw4yTv8BdR3H",
      "evaluationSetId": "ev_2LcgeY_mp2T5yPaEuq5Lw",
      "file": {
        "object": "file",
        "id": "file_xK9mLPqRtN3vS8wF5hB2cQ",
        "name": "Invoices.pdf",
        "type": "PDF",
        "parentFileId": "file_Zk9mNP12Qw4yTv8BdR3H",
        "metadata": {
          "pageCount": 30,
          "parentSplit": {
            "id": "string",
            "type": "Invoice",
            "identifier": "other_2_9",
            "startPage": 1,
            "endPage": 10
          }
        },
        "createdAt": "2024-03-21T16:45:00Z",
        "updatedAt": "2024-03-21T16:45:00Z"
      },
      "files": [
        {
          "object": "file",
          "id": "file_xK9mLPqRtN3vS8wF5hB2cQ",
          "name": "Invoices.pdf",
          "type": "PDF",
          "parentFileId": "file_Zk9mNP12Qw4yTv8BdR3H",
          "metadata": {
            "pageCount": 30,
            "parentSplit": {
              "id": "string",
              "type": "Invoice",
              "identifier": "other_2_9",
              "startPage": 1,
              "endPage": 10
            }
          },
          "createdAt": "2024-03-21T16:45:00Z",
          "updatedAt": "2024-03-21T16:45:00Z"
        }
      ],
      "expectedOutput": {
        "value": {}
      },
      "createdBy": {
        "type": "user",
        "email": "matt@extend.app"
      }
    }
  ]
}

Evaluation set items are the individual files and expected outputs that are used to evaluate the performance of a given extractor, classifier, or splitter in Extend. This endpoint will create new evaluation set items in Extend, which will be used during an evaluation run.

Each item references exactly one of:

fileId — a single file. The item is evaluated as one single-file run.
fileIds — an ordered package of 2–50 files. The item is evaluated as one multifile extraction run, with a single expectedOutput covering the whole package. Multifile items are only supported on evaluation sets attached to a JSON-schema extractor.

Limit: You can create up to 100 items at a time.

Learn more about how to create evaluation set items in the Evaluation Sets product page.

Authentication

AuthorizationBearer

Bearer authentication of the form Bearer <token>, where token is your auth token.

Path parameters

evaluationSetIdstringRequired

The ID of the evaluation set.

Example: "ev_2LcgeY_mp2T5yPaEuq5Lw"

Headers

Request

This endpoint expects an object.

itemslist of objectsRequired

An array of objects representing the evaluation set items to create.

Response

Evaluation set items created successfully

evaluationSetItemslist of objects

Errors

400

Bad Request Error

401

Unauthorized Error

402

Payment Required Error

403

Forbidden Error

404

Not Found Error

422

Unprocessable Entity Error

429

Too Many Requests Error

500

Internal Server Error