Migration Guides2026-02-09

Evaluation Sets Migration

What’s Changed

  • New evaluation set runs endpoints — POST /evaluation_set_runs to start an evaluation run and GET /evaluation_set_runs/{id} to retrieve it
  • Nested item paths — /evaluation_set_items → /evaluation_sets/{id}/items
  • New get item endpoint — GET /evaluation_sets/{id}/items/{itemId} (wasn’t available before)
  • Entity objects — processorId replaced with typed entity object

If you don’t use evaluation sets, you can skip this page.

Quick Start: Common Patterns

Before (2025-04-21)
1// Evaluation Sets
2const sets = await client.evaluationSet.list({ processorId: "dp_abc123" });
3const set = await client.evaluationSet.get("ev_abc123");
4
5// Evaluation Set Items
6const items = await client.evaluationSetItem.list("ev_abc123");
7await client.evaluationSetItem.update("evi_xyz789", { expectedOutput: {...} });
After (2026-02-09)
1// Evaluation Set Runs (new!)
2const evalRun = await client.evaluationSetRuns.retrieve("bpr_abc123");
3
4// Evaluation Sets
5const sets = await client.evaluationSets.list({ entityId: "ex_abc123" });
6const set = await client.evaluationSets.retrieve("ev_abc123");
7
8// Evaluation Set Items
9const items = await client.evaluationSetItems.list("ev_abc123");
10const item = await client.evaluationSetItems.retrieve("ev_abc123", "evi_xyz789");
11await client.evaluationSetItems.update("ev_abc123", "evi_xyz789", { expectedOutput: {...} });

Endpoint Changes Summary

Old EndpointNew EndpointChange
GET /batch_processor_runs/{id}GET /batch_processor_runs/{id}Deprecated but still works
—POST /evaluation_set_runsNew for starting evaluation set runs
—GET /evaluation_set_runs/{id}New for evaluation set runs
GET /evaluation_setsGET /evaluation_setsQuery param processorId → entityId
POST /evaluation_setsPOST /evaluation_setsRequest body processorId → entityId
GET /evaluation_sets/{id}GET /evaluation_sets/{id}Response has entity instead of processorId
GET /evaluation_sets/{id}/itemsGET /evaluation_sets/{id}/itemsReturns summaries (no expectedOutput)
—GET /evaluation_sets/{id}/items/{itemId}New for full item details
POST /evaluation_set_itemsPOST /evaluation_sets/{id}/itemsPath changed, supports batch
POST /evaluation_set_items/bulkPOST /evaluation_sets/{id}/itemsConsolidated into single endpoint
POST /evaluation_set_items/{id}POST /evaluation_sets/{id}/items/{itemId}Path changed
DELETE /evaluation_set_items/{id}DELETE /evaluation_sets/{id}/items/{itemId}Path changed

Request Changes

Evaluation Sets

OldNewNotes
processorId (query/body)entityIdRenamed

Evaluation Set Items

OldNewNotes
POST /evaluation_set_items with evaluationSetId in bodyPOST /evaluation_sets/{id}/itemsID moved to path
Separate single/bulk endpointsUnified endpoint (1-100 items)Use items array

Response Changes

Response shape changes: Single object responses are now returned directly (no wrapper key), and list responses use { "object": "list", "data": [...] } format. See Simplified Response Shapes for details.

Key Differences

OldNew
success: true(removed) — Use HTTP status codes
{ "evaluationSet": {...} }{...} (object returned directly)
processorIdentity object with full details
fileIdfile object with full details
List items returns full objectsList returns summaries (use retrieve for full)

Example: EvaluationSet Response

After (2026-02-09)
1{
2  "object": "evaluation_set",
3  "id": "ev_abc123",
4  "name": "Invoice Test Set",
5  "entity": {
6    "object": "extractor",
7    "id": "ex_xyz789",
8    "name": "Invoice Extractor"
9  },
10  "createdAt": "2024-03-21T15:30:00Z",
11  "updatedAt": "2024-03-21T16:45:00Z"
12}

SDK Method Reference

Old MethodNew Method
client.batchProcessorRun.get()client.batchProcessorRuns.retrieve() (deprecated)
—client.evaluationSetRuns.retrieve() (new)
client.evaluationSet.list()client.evaluationSets.list()
client.evaluationSet.get()client.evaluationSets.retrieve()
client.evaluationSet.create()client.evaluationSets.create()
client.evaluationSetItem.list()client.evaluationSetItems.list()
—client.evaluationSetItems.retrieve() (new)
client.evaluationSetItem.create()client.evaluationSetItems.create()
client.evaluationSetItem.createBatch()client.evaluationSetItems.create() (supports arrays)
client.evaluationSetItem.update()client.evaluationSetItems.update()
client.evaluationSetItem.delete()client.evaluationSetItems.delete()

EvaluationSetRun Schema (New)

PropertyOld (BatchProcessorRun)New (EvaluationSetRun)Change
object"batch_processor_run""evaluation_set_run"Value changed
idRequired stringRequired stringNo change
evaluationSetIdsourceId fieldRequired stringRenamed
processorIdRequired—Removed, see entity.id
entity—RequiredNew
entityVersion—RequiredNew
sourceRequired "EVAL_SET"—Removed (implicit)
runCountRequired—Removed

EvaluationSet Schema

PropertyOldNewChange
processorIdRequired string—Removed
entity—RequiredNew (replaces processorId)

EvaluationSetItem Schema

PropertyOldNewChange
fileIdRequired string—Removed
file—Required FileSummaryNew (replaces fileId)

EvaluationSetItemSummary Schema (New)

Returned by list endpoint instead of full items:

PropertyType
object"evaluation_set_item"
idRequired string
fileRequired FileSummary

Need Help?

If you encounter any issues while migrating, please contact our support team at support@extend.app.

Migration Guides

GuideMigrating FromMigrating To
Overview—What’s new and how to upgrade
Extract Runs/processor_runs/extract_runs + /extract
Classify Runs/processor_runs/classify_runs + /classify
Split Runs/processor_runs/split_runs + /split
Parse Runs/parse, /parse/async/parse_runs + /parse
Edit Runs/edit, /edit/async/edit_runs + /edit
Extractors/processors/extractors
Classifiers/processors/classifiers
Splitters/processors/splitters
Files/files/files (breaking changes)
Evaluation Setsevaluation endpointsUpdated evaluation endpoints
Workflow Runs/workflow_runs/workflow_runs (breaking changes)
Webhooksprocessor_run.* eventsextract_run.*, classify_run.*, etc.