Bounding Boxes (Legacy)

In addition to displaying citations in the UI, we include location references (when available) in the API response payload for workflow run outputs. The format of these references depends on your processor’s configuration type.

Citations are used for the modern JSON Schema configuration, while Bounding Boxes are used for the legacy Fields Array configuration. Extractors made after April 2025 are typically on the JSON Schema configuration.

Importantly, these references are not the same as traditional OCR and we do not make a guarantee that 100% of the time our system is able to detect and return them for all extracted values. However, we are constantly improving our models and will continue to do so.

Right now location references are only available for Extract output fields, and are only supported for the following file/document types:

• PDF
• IMG (jpeg, png, etc)

Default Bounding Boxes

The default bounding box feature uses heuristic-based matches and supports the following field types:

• date fields
• string fields
• signature fields
• array fields (on nested string fields)
• object fields (on nested string fields)

Advanced Bounding Boxes

If you have selected “Advanced bounding box” in the extraction settings in the Extend Studio, bounding boxes can be provided for additional field types with potentially higher coverage:

• enum fields
• number fields
• boolean fields
• null fields - If a field is declaratively null (e.g., an empty form input), a bounding box reference may be returned. If there is no declarative indication of null, bounding boxes will not be returned.

You can toggle this on in the Advanced Settings of an extraction configuration in Extend Studio:

• Schema: Bounding Boxes use a left, top, right, bottom structure.

How to use bounding boxes

How to use the bounding values in order to place a bounding box on a file, depends heavily on the file type. In general though, you will need to take the bounding box values we return and convert them to whatever coordinate system your rendering library uses or what you define if you are using a native Canvas element approach to drawing them over an image for instance.

The values we return in the BoundingBox type represent the coordinates of the bounding box on the image or page. They indicate the top, bottom, left, and right extremities of the box. You may need to convert these values into a format suitable for your rendering library. If your rendering library uses a coordinate system based on percentages of the total image or page size, you would need to perform an additional conversion step.

For PDFs, here is an example of how to apply a transform to the value in order to be rendered using react-pdf-viewer or a similar library:

1
export type HighlightArea = {
2
  left: number;
3
  top: number;
4
  width: number;
5
  height: number;
6
  pageIndex: number;
7
};
8
9
function convertBoundingBoxToHighlightArea({
10
  boundingBox,
11
  pageIndex,
12
  pageHeight,
13
  pageWidth,
14
}: {
15
  boundingBox: BoundingBox;
16
  pageIndex: number; // The index of the page in the document (0 indexed)
17
  pageHeight: number;
18
  pageWidth: number;
19
}): HighlightArea {
20
  // If you are able to access the page height and width of the document, use them to calculate the highlight area
21
  if (pageHeight && pageWidth) {
22
    const highlightArea: HighlightArea = {
23
      height: ((boundingBox.bottom - boundingBox.top) / pageHeight) * 100 + 2,
24
      left: (boundingBox.left / pageWidth) * 100 - 1,
25
      width: ((boundingBox.right - boundingBox.left) / pageWidth) * 100 + 2,
26
      top: (boundingBox.top / pageHeight) * 100 - 1,
27
      pageIndex,
28
    };
29
    return highlightArea;
30
  }
31
32
  const DPI = 72; // Default DPI - calibrate this based on your product
33
34
  // Otherwise you can default to a conversion based on the zoom level of the document
35
  const highlightArea: HighlightArea = {
36
    left: boundingBox.left * 9,
37
    top: boundingBox.top * 9,
38
    width: ((boundingBox.right - boundingBox.left) * DPI) / 8,
39
    height: ((boundingBox.bottom - boundingBox.top) * DPI) / 8,
40
    pageIndex,
41
  };
42
  return highlightArea;
43
}