Bulk Create Evaluation Set Items

Create evaluation set items for a given evaluation set in bulk.

1 curl --request POST \
2   --url https://api-prod.extend.app/evaluation_set_items/bulk \
3   --header 'Authorization: Bearer <token>' \
4   --header 'Content-Type: application/json' \
5   --data '{
6   "evaluationSetId": "<string>",
7   "items": [
8     {}
9   ]
10 }'

Example Response

1 {
2   "success": true,
3   "evaluationSetItems": [
4     {
5       "object": "evaluation_set_item",
6       "id": "eval_item_1234",
7       "evaluationSetId": "eval_set_1234",
8       "fileId": "file_1234",
9       "expectedOutput": {
10         // Output object - see “Processor output types” for details
11       },
12       "createdAt": "2024-01-01T00:00:00Z",
13       "updatedAt": "2024-01-01T00:00:00Z"
14     },
15     {
16       "object": "evaluation_set_item",
17       "id": "eval_item_5678",
18       "evaluationSetId": "eval_set_5678",
19       "fileId": "file_5678",
20       "expectedOutput": {
21         // Output object - see “Processor output types” for details
22       },
23       "createdAt": "2024-01-01T00:00:00Z",
24       "updatedAt": "2024-01-01T00:00:00Z"
25     }
26   ]
27 }

If you have a large number of files that you need to add to an evaluation set, you can use this endpoint to create multiple evaluation set items at once. This can be useful if you have a large dataset that you need to evaluate the performance of a processor against.

Note: you still need to create each File first using the file API.

Body

evaluationSetId

stringRequired

The ID of the evaluation set to add the items to.

items

arrayRequired

An array of objects representing the evaluation set items to create. Each object should have the following fields:

properties

fileId

string

The ID of the file to add to the evaluation set.

expectedOutput

object

The expected output of the processor when run against the file. This should be a JSON object conforming to the output type schema of the processor.

Response

boolean

A true or false value for whether the evaluation set items were created successfully or not.

array

An array of EvaluationSetItem objects representing the newly created evaluation set items. See the EvaluationSetItem object for more details.