Core Document ProcessingExtraction

Extractor output type

Extraction outputs follow a standardized format that you’ll encounter when working with evaluation sets, webhooks, and API responses.

Extraction output type

The output structure for JSON Schema processors is composed of two properties: value and metadata.

The value object is the actual data extracted from the document which conforms to the JSON Schema defined in the processor config.

The metadata object holds details like confidence scores and citations for the extracted data. It uses keys that represent the path to the corresponding data within the value object. Crucially, the keys in the metadata object mirror the structure of the value object using a path-like notation (e.g., line_items[0].description), allowing you to precisely pinpoint metadata for any specific field, including those nested within objects or arrays. For instance, if your data has value.line_items[0].name, the metadata specifically for that name field will be found using the key ‘line_items[0].name’ within the metadata object.

The extraction_light base processor does not support logprobs. For JSON Schema processors, this means metadata entries will have logprobsConfidence set to null.

Type definition

TypeScript Types
1type ExtractionOutput = {
2  value: ExtractionValue;
3  metadata: ExtractionMetadata;
4};
5
6type ExtractionValue = Record<string, any>; // Conforms to the schema defined in the processor config
7type ExtractionMetadata = {
8  [key: string]: ExtractionMetadataEntry | undefined;
9};
10
11type ExtractionMetadataEntry {
12  ocrConfidence?: number | null;
13  logprobsConfidence: number | null;
14  reviewAgentScore?: number | null;
15  citations?: Citation[];
16  insights?: OutputInsight[];
17}
18
19type Citation = {
20  page: {
21    number: number;
22    width: number;
23    height: number;
24  };
25  referenceText?: string | null;
26  polygon?: Point[];
27};
28
29type Point = {
30  x: number;
31  y: number;
32};
33
34type OutputInsight =
35  | { type: "reasoning"; content: string }
36  | { type: "issue"; content: string }
37  | { type: "review_summary"; content: string };

The insights array is a shared channel. It can include reasoning (when model reasoning insights are enabled) and issue/review_summary (when the Review Agent is enabled). See Review Agent for more details.

Accessing Metadata

To access the metadata for a specific field, especially nested ones like items in an array, you use a path-like key string. For example, to get the metadata for the description of the first item in a line_items array, the key would be line_items[0].description.

Here are examples in Python and TypeScript:

1const output = {
2  value: {
3    invoice_number: "INV-123",
4    line_items: [
5      { description: "Item A", quantity: 2, price: 10.0 },
6      { description: "Item B", quantity: 1, price: 25.5 },
7    ],
8  },
9  metadata: {
10    invoice_number: {
11      logprobsConfidence: 1,
12      ocrConfidence: 0.99,
13      citations: [
14        {
15          referenceText: "Invoice #: INV-123",
16          page: { number: 1, width: 612, height: 792 },
17          polygon: [
18            { x: 296.73359999999997, y: 40.888799999999996 },
19            { x: 386.4168, y: 40.464000000000006 },
20            { x: 386.4744, y: 52.1712 },
21            { x: 296.7912, y: 52.596000000000004 },
22          ],
23        },
24      ],
25    },
26    line_items: {
27      logprobsConfidence: 0.98,
28      ocrConfidence: 0.98,
29    },
30    "line_items[0]": {
31      logprobsConfidence: 0.98,
32      ocrConfidence: 0.98,
33      citations: [{ page: { number: 1, width: 612, height: 792 } }],
34    },
35    "line_items[0].description": { logprobsConfidence: 1, ocrConfidence: 0.95 },
36    "line_items[0].quantity": { logprobsConfidence: 1, ocrConfidence: 0.98 },
37    "line_items[0].price": { logprobsConfidence: 1, ocrConfidence: 0.98 },
38    "line_items[1].description": { logprobsConfidence: 1, ocrConfidence: 0.96 },
39    // Other metadata entries...
40  },
41};
42
43// Traversing the output object and accessing metadata
44const invoiceNumber = output.value.invoice_number;
45const invoiceNumberMetadata = output.metadata.invoice_number;
46
47// Access metadata for the line_items array itself
48const lineItemsMetadata = output.metadata.line_items;
49
50// Loop through line items array
51for (let i = 0; i < output.value.line_items.length; i++) {
52  const lineItemPath = `line_items[${i}]`;
53
54  // Access the line item object and its metadata
55  const lineItem = output.value.line_items[i];
56  const lineItemMetadata = output.metadata[lineItemPath];
57
58  // Access properties within the line item and their metadata
59  const lineItemDescription = lineItem.description;
60  const lineItemDescriptionMetadata =
61    output.metadata[`${lineItemPath}.description`];
62  const lineItemQuantity = lineItem.quantity;
63  const lineItemQuantityMetadata = output.metadata[`${lineItemPath}.quantity`];
64  const lineItemPrice = lineItem.price;
65  const lineItemPriceMetadata = output.metadata[`${lineItemPath}.price`];
66}

Examples

1{
2  "value": {
3    "amount": {
4      "amount": 15735.1,
5      "iso_4217_currency_code": "USD"
6    },
7    "invoice_number": "36995"
8  },
9  "metadata": {
10    "amount": {
11      "insights": [
12        {
13          "type": "reasoning",
14          "content": "The total amount is shown as '$15,735.1' in both the table summary and the bottom right of the document. The currency symbol '$' and the US address indicate the currency is USD. The value is numeric and matches the required format."
15        }
16      ],
17      "citations": [
18        {
19          "page": {
20            "number": 1,
21            "width": 612,
22            "height": 792
23          },
24          "polygon": [
25            {
26              "x": 430.164,
27              "y": 722.772
28            },
29            {
30              "x": 467.27279999999996,
31              "y": 722.8296
32            },
33            {
34              "x": 467.2584,
35              "y": 731.6351999999999
36            },
37            {
38              "x": 430.1496,
39              "y": 731.5776
40            }
41          ],
42          "referenceText": "TOTAL  $15,735.1"
43        }
44      ],
45      "ocrConfidence": 0.992,
46      "logprobsConfidence": 1
47    },
48    "invoice_number": {
49      "insights": [
50        {
51          "type": "reasoning",
52          "content": "The invoice number is clearly labeled as 'Invoice #36995' at the top right of the document, making it straightforward to extract."
53        }
54      ],
55      "citations": [
56        {
57          "page": {
58            "number": 1,
59            "width": 612,
60            "height": 792
61          },
62          "polygon": [
63            {
64              "x": 296.73359999999997,
65              "y": 40.888799999999996
66            },
67            {
68              "x": 386.4168,
69              "y": 40.464000000000006
70            },
71            {
72              "x": 386.4744,
73              "y": 52.1712
74            },
75            {
76              "x": 296.7912,
77              "y": 52.596000000000004
78            }
79          ],
80          "referenceText": "Invoice #36995"
81        }
82      ],
83      "ocrConfidence": 0.986,
84      "logprobsConfidence": 1
85    }
86  }
87}
1{
2  "value": {
3    "line_items": [
4      {
5        "item": "Widget A",
6        "quantity": 5,
7        "price": {
8          "amount": 10.0,
9          "iso_4217_currency_code": "USD"
10        }
11      },
12      {
13        "item": "Widget B",
14        "quantity": 2,
15        "price": {
16          "amount": 15.0,
17          "iso_4217_currency_code": "USD"
18        }
19      }
20    ],
21    "signature_block": {
22      "printed_name": "John Smith",
23      "signature_date": "2024-03-15",
24      "is_signed": true,
25      "title_or_role": "Purchasing Manager"
26    }
27  },
28  "metadata": {
29    "line_items": {
30      "logprobsConfidence": 0.96 // Minimum confidence for the values in the array
31    },
32    "line_items[0]": {
33      "logprobsConfidence": 0.96, // Minimum confidence for the values in the item
34      "citations": [{ "page": { "number": 1, "width": 612, "height": 792 } }]
35    },
36    "line_items[0].item": {
37      "logprobsConfidence": 0.99
38    },
39    "line_items[0].quantity": {
40      "logprobsConfidence": 1.0
41    },
42    "line_items[0].price.amount": {
43      "logprobsConfidence": 0.98
44    },
45    "line_items[0].price.iso_4217_currency_code": {
46      "logprobsConfidence": 0.96
47    },
48    "line_items[1]": {
49      "logprobsConfidence": 0.96,
50      "citations": [{ "page": { "number": 1, "width": 612, "height": 792 } }]
51    },
52    "line_items[1].item": {
53      "logprobsConfidence": 0.98
54    },
55    "line_items[1].quantity": {
56      "logprobsConfidence": 1.0
57    },
58    "line_items[1].price.amount": {
59      "logprobsConfidence": 0.97
60    },
61    "line_items[1].price.iso_4217_currency_code": {
62      "logprobsConfidence": 0.96
63    },
64    "signature_block": {
65      "logprobsConfidence": 0.99, // Confidence for the overall object
66      "citations": [
67        {
68          "page": { "number": 1, "width": 612, "height": 792 },
69          "referenceText": "John Smith",
70          "polygon": [
71            /* points omitted */
72          ]
73        }
74      ]
75    }
76  }
77}

Shared Types

Certain types are shared across different processor outputs. These provide additional context and information about the processor’s decisions.

Type Definition

1type Insight = {
2  type: "reasoning" | "issue" | "review_summary";
3  content: string;
4};

Example

1{
2  "insights": [
3    {
4      "type": "reasoning",
5      "content": "This was classified as an invoice because it contains standard invoice elements including an invoice number, billing details, and itemized charges."
6    }
7  ]
8}

Insights can appear in both Extraction and Classification outputs to provide transparency into the model’s decision-making process. They are particularly useful when debugging or validating processor results.

  • reasoning: Model explanations when reasoning insights are enabled
  • issue: Problems identified by the Review Agent
  • review_summary: Overall assessment from the Review Agent