Extractor output type

Document processor outputs follow standardized formats based on the processor type. Understanding these formats is essential when working with evaluation sets, webhooks, and API responses.

Extraction output type

The output structure for JSON Schema processors is composed of two properties: value and metadata.

The value object is the actual data extracted from the document which conforms to the JSON Schema defined in the processor config.

The metadata object holds details like confidence scores and citations for the extracted data. It uses keys that represent the path to the corresponding data within the value object. Crucially, the keys in the metadata object mirror the structure of the value object using a path-like notation (e.g., line_items[0].description), allowing you to precisely pinpoint metadata for any specific field, including those nested within objects or arrays. For instance, if your data has value.line_items[0].name, the metadata specifically for that name field will be found using the key ‘line_items[0].name’ within the metadata object.

Type definition

TypeScript Types

1 type ExtractionOutput = {
2   value: ExtractionValue;
3   metadata: ExtractionMetadata;
4 };
5 
6 type ExtractionValue = Record<string, any>; // Conforms to the schema defined in the processor config
7 type ExtractionMetadata = {
8   [key: string]: ExtractionMetadataEntry | undefined;
9 };
10 
11 type ExtractionMetadataEntry {
12   ocrConfidence?: number | null;
13   logprobsConfidence: number | null;
14   citations?: Citation[];
15   insights?: OutputInsight[];
16 }
17 
18 type Citation = {
19   page?: number;
20   referenceText?: string | null;
21   polygon?: Point[];
22 };
23 
24 type Point = {
25   x: number;
26   y: number;
27 };
28 
29 type OutputInsight = {
30   type: "reasoning";
31   content: string;
32 };

Accessing Metadata

To access the metadata for a specific field, especially nested ones like items in an array, you use a path-like key string. For example, to get the metadata for the description of the first item in a line_items array, the key would be line_items[0].description.

Here are examples in Python and TypeScript:

1 const output = {
2   value: {
3     invoice_number: "INV-123",
4     line_items: [
5       { description: "Item A", quantity: 2, price: 10.0 },
6       { description: "Item B", quantity: 1, price: 25.5 },
7     ],
8   },
9   metadata: {
10     invoice_number: {
11       logprobsConfidence: 1,
12       ocrConfidence: 0.99,
13       citations: [
14         {
15           referenceText: "Invoice #: INV-123",
16           page: 1,
17           polygon: [
18             { x: 296.73359999999997, y: 40.888799999999996 },
19             { x: 386.4168, y: 40.464000000000006 },
20             { x: 386.4744, y: 52.1712 },
21             { x: 296.7912, y: 52.596000000000004 },
22           ],
23         },
24       ],
25     },
26     line_items: {
27       logprobsConfidence: 0.98,
28       ocrConfidence: 0.98,
29     },
30     "line_items[0]": {
31       logprobsConfidence: 0.98,
32       ocrConfidence: 0.98,
33       citations: [{ page: 1 }],
34     },
35     "line_items[0].description": { logprobsConfidence: 1, ocrConfidence: 0.95 },
36     "line_items[0].quantity": { logprobsConfidence: 1, ocrConfidence: 0.98 },
37     "line_items[0].price": { logprobsConfidence: 1, ocrConfidence: 0.98 },
38     "line_items[1].description": { logprobsConfidence: 1, ocrConfidence: 0.96 },
39     // Other metadata entries...
40   },
41 };
42 
43 // Traversing the output object and accessing metadata
44 const invoiceNumber = output.value.invoice_number;
45 const invoiceNumberMetadata = output.metadata.invoice_number;
46 
47 // Access metadata for the line_items array itself
48 const lineItemsMetadata = output.metadata.line_items;
49 
50 // Loop through line items array
51 for (let i = 0; i < output.value.line_items.length; i++) {
52   const lineItemPath = `line_items[${i}]`;
53 
54   // Access the line item object and its metadata
55   const lineItem = output.value.line_items[i];
56   const lineItemMetadata = output.metadata[lineItemPath];
57 
58   // Access properties within the line item and their metadata
59   const lineItemDescription = lineItem.description;
60   const lineItemDescriptionMetadata =
61     output.metadata[`${lineItemPath}.description`];
62   const lineItemQuantity = lineItem.quantity;
63   const lineItemQuantityMetadata = output.metadata[`${lineItemPath}.quantity`];
64   const lineItemPrice = lineItem.price;
65   const lineItemPriceMetadata = output.metadata[`${lineItemPath}.price`];
66 }

Examples

Basic Field Types

1 {
2   "value": {
3     "amount": {
4       "amount": 15735.1,
5       "iso_4217_currency_code": "USD"
6     },
7     "invoice_number": "36995"
8   },
9   "metadata": {
10     "amount": {
11       "insights": [
12         {
13           "type": "reasoning",
14           "content": "The total amount is shown as '$15,735.1' in both the table summary and the bottom right of the document. The currency symbol '$' and the US address indicate the currency is USD. The value is numeric and matches the required format."
15         }
16       ],
17       "citations": [
18         {
19           "page": 1,
20           "polygon": [
21             {
22               "x": 430.164,
23               "y": 722.772
24             },
25             {
26               "x": 467.27279999999996,
27               "y": 722.8296
28             },
29             {
30               "x": 467.2584,
31               "y": 731.6351999999999
32             },
33             {
34               "x": 430.1496,
35               "y": 731.5776
36             }
37           ],
38           "referenceText": "TOTAL  $15,735.1"
39         }
40       ],
41       "ocrConfidence": 0.992,
42       "logprobsConfidence": 1
43     },
44     "invoice_number": {
45       "insights": [
46         {
47           "type": "reasoning",
48           "content": "The invoice number is clearly labeled as 'Invoice #36995' at the top right of the document, making it straightforward to extract."
49         }
50       ],
51       "citations": [
52         {
53           "page": 1,
54           "polygon": [
55             {
56               "x": 296.73359999999997,
57               "y": 40.888799999999996
58             },
59             {
60               "x": 386.4168,
61               "y": 40.464000000000006
62             },
63             {
64               "x": 386.4744,
65               "y": 52.1712
66             },
67             {
68               "x": 296.7912,
69               "y": 52.596000000000004
70             }
71           ],
72           "referenceText": "Invoice #36995"
73         }
74       ],
75       "ocrConfidence": 0.986,
76       "logprobsConfidence": 1
77     }
78   }
79 }

Nested Structures

1 {
2   "value": {
3     "line_items": [
4       {
5         "item": "Widget A",
6         "quantity": 5,
7         "price": {
8           "amount": 10.0,
9           "iso_4217_currency_code": "USD"
10         }
11       },
12       {
13         "item": "Widget B",
14         "quantity": 2,
15         "price": {
16           "amount": 15.0,
17           "iso_4217_currency_code": "USD"
18         }
19       }
20     ],
21     "signature_block": {
22       "printed_name": "John Smith",
23       "signature_date": "2024-03-15",
24       "is_signed": true,
25       "title_or_role": "Purchasing Manager"
26     }
27   },
28   "metadata": {
29     "line_items": {
30       "logprobsConfidence": 0.96 // Minimum confidence for the values in the array
31     },
32     "line_items[0]": {
33       "logprobsConfidence": 0.96, // Minimum confidence for the values in the item
34       "page": 1
35     },
36     "line_items[0].item": {
37       "logprobsConfidence": 0.99
38     },
39     "line_items[0].quantity": {
40       "logprobsConfidence": 1.0
41     },
42     "line_items[0].price.amount": {
43       "logprobsConfidence": 0.98
44     },
45     "line_items[0].price.iso_4217_currency_code": {
46       "logprobsConfidence": 0.96
47     },
48     "line_items[1]": {
49       "logprobsConfidence": 0.96,
50       "page": 1
51     },
52     "line_items[1].item": {
53       "logprobsConfidence": 0.98
54     },
55     "line_items[1].quantity": {
56       "logprobsConfidence": 1.0
57     },
58     "line_items[1].price.amount": {
59       "logprobsConfidence": 0.97
60     },
61     "line_items[1].price.iso_4217_currency_code": {
62       "logprobsConfidence": 0.96
63     },
64     "signature_block": {
65       "logprobsConfidence": 0.99, // Confidence for the overall object
66       "citations": [
67         {
68           "page": 1,
69           "referenceText": "John Smith",
70           "polygon": [
71             /* points omitted */
72           ]
73         }
74       ]
75     }
76   }
77 }

Shared Types

Certain types are shared across different processor outputs. These provide additional context and information about the processor’s decisions.

Type Definition

1 type Insight = {
2   type: "reasoning"; // Currently only reasoning is supported
3   content: string; // The explanation or reasoning provided by the model
4 };

Example

1 {
2   "insights": [
3     {
4       "type": "reasoning",
5       "content": "This was classified as an invoice because it contains standard invoice elements including an invoice number, billing details, and itemized charges."
6     }
7   ]
8 }

Insights can appear in both Extraction and Classification outputs to provide transparency into the model’s decision-making process. They are particularly useful when debugging or validating processor results.