Document Extraction

Schema (JSON Schema)

JSON Schema Structure (schema)

We use JSON Schema to define the structure of the data we extract. Before you get started, we recommend familiarizing yourself with the JSON Schema documentation to understand how to define your schema.

The standard JSON Schema is extremely flexible. We’ve implemented a subset of the standard to support the needs of document extraction. Your schema must follow these rules:

  • The root must be an object type
  • Allowed types are string, number, integer, boolean, object, and array
  • All primitive fields (string, number, boolean, integer) must be nullable (use array type with “null” as an option e.g. "type": ["string", "null"]). If you do not provide a “null” option, it will be automatically added.
  • Maximum nesting level is 3 (each non-root object counts as 1 level)
  • Property keys and names can contain letters, numbers, underscores, and hyphens
  • Array items must be objects
  • Enums must only contain strings and must contain a null option. If you do not provide a null option, it will be automatically added.
  • Custom types are supported by adding a "extend:type": "currency", "extend:type": "signature", or "extend:type": "date" property to the appropriate field type with the required properties. See below for examples.
  • Property names can be added using the "extend:name" property. If supplied, this will override the name of the property as it will appear to the model, but not in the output returned to you. This is useful for providing more descriptive names or instructions to the model without altering the actual keys in your output data structure.
  • You can add descriptions to individual enum values using the "extend:descriptions" property.

Unsupported Features

While we support the JSON Schema structure, we do not support many of the additional features some of which include:

Schema Examples

Primitive Schema

All primitive types must be nullable.

1{
2  "field_name": {
3    "type": ["string", "null"],
4    "description": "Field description"
5  },
6  "numeric_field": {
7    "type": ["number", "null"],
8    "description": "A numeric field"
9  },
10  "integer_field": {
11    "type": ["integer", "null"],
12    "description": "An integer field"
13  },
14  "boolean_field": {
15    "type": ["boolean", "null"],
16    "description": "A boolean field"
17  }
18}

Object Schema

Objects must have properties. If you set a required array of the properties, we will respect that order when extracting. If you do not set required array, we will generate it and enforce order.

1{
2  "address": {
3    "type": "object",
4    "properties": {
5      "street": {
6        "type": ["string", "null"],
7        "description": "Street address"
8      },
9      "city": {
10        "type": ["string", "null"],
11        "description": "City name"
12      }
13    },
14    "required": ["street", "city"]
15  }
16}

Array Schema

Arrays items must be objects.

1{
2  "line_items": {
3    "type": "array",
4    "items": {
5      "type": "object",
6      "properties": {
7        "description": {
8          "type": ["string", "null"],
9          "description": "Item description"
10        },
11        "quantity": {
12          "type": ["number", "null"],
13          "description": "Item quantity"
14        },
15        "price": {
16          "type": ["number", "null"],
17          "description": "Item price"
18        }
19      },
20      "required": ["description", "quantity", "price"]
21    },
22    "description": "List of items in the invoice"
23  }
24}

Enum Schema

Enums must include null as an option. Only strings are supported for enums. The extend:descriptions is an optional array of strings. It is recommended to give more context for each enum option for more accurate extraction.

1{
2  "status": {
3    "enum": ["pending", "approved", "rejected", null],
4    "extend:descriptions": [
5      "Invoice is pending approval",
6      "Invoice has been approved",
7      "Invoice has been rejected",
8      ""
9    ],
10    "description": "Current status of the invoice"
11  }
12}

Custom Field Types

The extend:type keyword enables custom pre-processing and post-processing of fields which bake in best practices and heuristics for the field type.

Date Schema

Date fields must be strings and use the extend:type keyword with the value date. This will guarantee the date format is always an ISO compliant date (yyyy-mm-dd).

1{
2  "invoice_date": {
3    "type": ["string", "null"],
4    "extend:type": "date",
5    "description": "The invoice date"
6  }
7}

Currency Schema

Currency fields must be objects with specific properties.

1{
2  "price": {
3    "type": "object",
4    "extend:type": "currency",
5    "properties": {
6      "amount": {
7        "type": ["number", "null"],
8      },
9      "iso_4217_currency_code": {
10        "type": ["string", "null"],
11      }
12    },
13    "required": ["amount", "iso_4217_currency_code"]
14  }
15}

Signature Schema

Signature fields must be objects with specific properties. This will auto-enable our advanced signature detection in the parsing step prior to extraction, and apply a number of prompt and post-processing heuristics to improve accuracy, particularly on reduction of false positives for signature blocks that are not actually signed.

1{
2  "signature": {
3    "type": "object",
4    "extend:type": "signature",
5    "properties": {
6      "printed_name": {
7        "type": ["string", "null"],
8      },
9      "signature_date": {
10        "type": ["string", "null"],
11        "extend:type": "date",
12      },
13      "is_signed": {
14        "type": ["boolean", "null"],
15      },
16      "title_or_role": {
17        "type": ["string", "null"],
18      }
19    },
20    "required": ["printed_name", "signature_date", "is_signed", "title_or_role"]
21  }
22}

Configuration Examples

1const basicExtractionConfig = {
2  type: "EXTRACT",
3  schema: {
4    type: "object",
5    properties: {
6      invoice_number: {
7        type: ["string", "null"],
8        description: "The unique identifier for this invoice",
9      },
10      invoice_amount: {
11        type: "object",
12        "extend:type": "currency",
13        description: "The total amount of the invoice",
14        properties: {
15          amount: {
16            type: ["number", "null"],
17          },
18          iso_4217_currency_code: {
19            type: ["string", "null"],
20          },
21        },
22        required: ["amount", "iso_4217_currency_code"],
23      },
24    },
25    required: ["invoice_number", "invoice_amount"],
26  },
27};
1const complexExtractionConfig = {
2  type: "EXTRACT",
3  schema: {
4    type: "object",
5    properties: {
6      line_items: {
7        type: "array",
8        description: "Individual items in the invoice",
9        items: {
10          type: "object",
11          properties: {
12            item_name: {
13              type: ["string", "null"],
14              description: "Name of the item",
15            },
16            quantity: {
17              type: ["number", "null"],
18              description: "Number of items",
19            },
20            unit_price: {
21              type: "object",
22              properties: {
23                amount: {
24                  type: ["number", "null"],
25                  description: "Price per unit",
26                },
27                iso_4217_currency_code: {
28                  type: ["string", "null"],
29                  description: "Currency code",
30                },
31              },
32              required: ["amount", "iso_4217_currency_code"],
33            },
34          },
35          required: ["item_name", "quantity", "unit_price"],
36        },
37      },
38      payment_status: {
39        description: "Current payment status",
40        enum: ["PAID", "PENDING", null],
41        "extend:descriptions": [
42          "Payment has been completed",
43          "Payment is pending",
44          "",
45        ],
46      },
47    },
48    required: ["line_items", "payment_status"],
49  },
50  customExtractionRules: "- If ...", // Optional custom rules
51  customDocumentKind: "invoice", // Optionally specify a document kind
52  includeBoundingBoxCitations: true, // Turns on llm-powered citations and bounding box references
53  includeModelReasoning: true, // Exposes the model's chain of thought reasoning for each field result
54};
1const nestedArrayConfig = {
2  type: "EXTRACT",
3  schema: {
4    type: "object",
5    properties: {
6      orders: {
7        type: "array",
8        description: "List of customer orders",
9        items: {
10          type: "object",
11          properties: {
12            order_id: {
13              type: ["string", "null"],
14              description: "Unique identifier for the order",
15            },
16            customer_name: {
17              type: ["string", "null"],
18              description: "Name of the customer",
19            },
20            shipments: {
21              type: "array",
22              description: "List of shipments for this order",
23              items: {
24                type: "object",
25                properties: {
26                  tracking_number: {
27                    type: ["string", "null"],
28                    description: "Shipping tracking number",
29                  },
30                  ship_date: {
31                    type: ["string", "null"],
32                    "extend:type": "date",
33                    description: "Date the shipment was sent",
34                  },
35                  carrier: {
36                    type: ["string", "null"],
37                    description: "Shipping carrier name",
38                  },
39                },
40                required: ["tracking_number", "ship_date", "carrier"],
41              },
42            },
43          },
44          required: ["order_id", "customer_name", "shipments"],
45        },
46      },
47    },
48    required: ["orders"],
49  },
50};
1const customFieldConfig = {
2  type: "EXTRACT",
3  schema: {
4    type: "object",
5    properties: {
6      invoice_signature: {
7        type: "object",
8        description: "Details of the invoice signature",
9        properties: {
10          printed_name: {
11            type: ["string", "null"],
12            description: "The printed name of the signer",
13          },
14          signature_date: {
15            type: ["string", "null"],
16            "extend:type": "date",
17            description: "The date the signature was applied",
18          },
19          is_signed: {
20            type: ["boolean", "null"],
21            description: "Indicates if the document is signed",
22          },
23          title_or_role: {
24            type: ["string", "null"],
25            description: "The title or role of the signer",
26          },
27        },
28        required: [
29          "printed_name",
30          "signature_date",
31          "is_signed",
32          "title_or_role",
33        ],
34      },
35      invoice_amount: {
36        type: "object",
37        "extend:type": "currency",
38        description: "The amount of the invoice",
39        properties: {
40          amount: {
41            type: ["number", "null"],
42            description: "The numerical value of the amount",
43          },
44          iso_4217_currency_code: {
45            type: ["string", "null"],
46            description: "The ISO 4217 currency code (e.g., USD, EUR)",
47          },
48        },
49        required: ["amount", "iso_4217_currency_code"],
50      },
51      invoice_date: {
52        type: "date",
53        description: "The date of the invoice",
54      },
55    },
56    required: ["signature", "invoice_amount", "invoice_date"],
57  },
58};

Type Definitions

1// Root schema
2type RootJSONSchema = {
3  type: "object";
4  properties: {
5    [key: string]: JSONSchema;
6  };
7  required: string[];
8  additionalProperties?: boolean;
9};
10
11// Common schema properties that all types can have
12type BaseJSONSchema = {
13  "extend:name"?: string;
14  description?: string;
15};
16
17// Enum schema
18type EnumJSONSchema = BaseJSONSchema & {
19  enum: (string | null)[]; // null will be automatically added if not provided
20  "extend:descriptions"?: string[];
21};
22
23// String schema
24type StringJSONSchema = BaseJSONSchema & {
25  type: ["string", "null"];
26  "extend:type"?: "date";
27};
28
29// Structure for a date field
30type DateStringSchema = BaseJSONSchema & {
31  type: ["string", "null"];
32  "extend:type": "date";
33};
34
35// Number schema
36type NumberJSONSchema = BaseJSONSchema & {
37  type: ["number", "null"];
38};
39
40// Integer schema
41type IntegerJSONSchema = BaseJSONSchema & {
42  type: ["integer", "null"];
43};
44
45// Boolean schema
46type BooleanJSONSchema = BaseJSONSchema & {
47  type: ["boolean", "null"];
48};
49
50// Array schema
51type ArrayJSONSchema = BaseJSONSchema & {
52  type: "array";
53  items: ObjectJSONSchema; // we only support objects in arrays for now
54};
55
56// Object schema
57type ObjectJSONSchema = BaseJSONSchema & {
58  type: "object";
59  properties: {
60    [key: string]: JSONSchema;
61  };
62  required: string[];
63  additionalProperties?: boolean;
64};
65
66// Required structure for a currency field
67type CurrencyObjectSchema = BaseJSONSchema & {
68  type: "object";
69  "extend:type": "currency";
70  properties: {
71    amount: {
72      type: ["number", "null"];
73    };
74    iso_4217_currency_code: {
75      type: ["string", "null"];
76    };
77  };
78  required: ["amount", "iso_4217_currency_code"];
79};
80
81// Required structure for a signature field
82type SignatureObjectSchema = BaseJSONSchema & {
83  type: "object";
84  "extend:type": "signature";
85  properties: {
86    printed_name: {
87      type: ["string", "null"];
88    };
89    signature_date: {
90      type: ["string", "null"];
91      "extend:type": "date"; // Note: The date itself also uses extend:type
92    };
93    is_signed: {
94      type: ["boolean", "null"];
95    };
96    title_or_role: {
97      type: ["string", "null"];
98    };
99  };
100  required: ["printed_name", "signature_date", "is_signed", "title_or_role"];
101};
102
103// Union of all schema types
104type JSONSchema =
105  | EnumJSONSchema
106  | StringJSONSchema
107  | NumberJSONSchema
108  | IntegerJSONSchema
109  | BooleanJSONSchema
110  | ArrayJSONSchema
111  | ObjectJSONSchema
112  | DateStringSchema
113  | CurrencyObjectSchema
114  | SignatureObjectSchema;