> This page is for version v2026-02-09 (default).
> For other versions, use one of these documentation indexes:
> - v2026-02-09 (default): https://docs.extend.ai/2026-02-09/llms.txt
> - v2025-04-21: https://docs.extend.ai/2025-04-21/llms.txt
> - v2024-12-23: https://docs.extend.ai/2024-12-23/llms.txt

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.extend.ai/llms.txt
> Use this file to discover all available pages before exploring further.
>
> ## API version
> The current API version is `2026-02-09`, served at the site root (no version prefix in URLs).
> If this page URL contains `/2025-04-21/` or `/2024-12-23/`, you are reading an older API version.
> Prefer the current docs at https://docs.extend.ai/llms.txt unless the user explicitly needs that older version.
> Do not treat older-version pages as the source of truth for new integrations.

# Review Agent

> The Review Agent analyzes extraction results to flag issues and produce an intelligent confidence score that improves on logprobs and OCR-based scores.

The Review Agent is a specialized system that analyzes extraction results to identify potential issues and produce an intelligent confidence score. It is designed to be a comprehensive replacement for both logprobs and OCR confidence scores, serving as a centralized metric to judge extraction confidence.

The Review Agent is designed to:

* Analyze extraction results for potential issues
* Produce an intelligent confidence score that is superior to standard logprobs and OCR-based scores
* Raise specific issues to help drive review and assist in making corrections

## Activating the Review Agent

You can activate the Review Agent in the Advanced settings for your extractor.

![](/_fern-img/f1f18cb46df4c4cb24823f4fecbf272ebc8b21dc8a5b0ca69c96084744054ff4.webp)

Once activated, the Review Agent will run on all subsequent extractions for this processor, providing review agent scores and issue reports in the extraction results.

## Review Agent UI

The Review Agent provides a dedicated interface for reviewing extraction confidence and issues.

### Extraction Confidence Gauge

The extraction confidence gauge gives you a high-level view of the overall confidence for a field. Hovering over the gauge reveals a detailed breakdown of the score and any specific issues found.

![](/_fern-img/3d379491cef6dcc94f387113d2c2cbe11460ed03ac714b38b9f24a4d54a4e40b.webp)

### Scalar Fields

For scalar fields (non-arrays), the interface shows a score and a review indicator. The tooltip expands to show specific issues raised for this item, along with a general review overview for that field.

![](/_fern-img/59b1cf8b1a4eb1c6b10c5cca418b0891bce41561277606d6165e8cdc830310f5.webp)

### Array Fields

The array interface provides detailed insights into list-based data. It displays common issues that appear across multiple items and includes a heatmap to help visualize where problems were detected in the extraction.

![](/_fern-img/2a75e8d92a78b68239fb13345e6a9e146e703e07d874b1cb439bcd0b64ba78a2.webp)

**Item-level Scoring:**
Each item in an array has its own score and issues, and arrays will have a new confidence column generated when running with the review agent enabled.

Hovering over the confidence cell for a specific item displays item-level issues, along with the score for that item.
Individual properties within an item also display their own scores when you hover over them. Cells with review scores ≤ 3 display a review indicator bar, helping you quickly identify where problems occur when scrolling through the array results.

![](/_fern-img/228d09540a5c5048d049cf5a41e509ae76d851eb0275f1ce4898fba4658c1417.webp)

## Scoring

The Review Agent produces a score that serves as a general indicator of extraction quality.

* **Score ≤ 3**: Usually warrants a review. The agent has likely found something of concern.
* **Score ≤ 4**: Consider reviewing if you are extremely sensitive to accuracy.
* **Lower Scores**: Scores below 3 typically indicate a higher number of major issues or issues of greater severity.

> **Note**
>
> OCR confidence is factored into the Review Agent score. Even if the agent does not identify semantic issues, a low OCR confidence can result in a lower overall score.

## Issues

Issues raised by the Review Agent are categorized by severity and relevance.

* **Severity Colors**: Issues are colored based on their severity.
  * **Red**: Severe problems that likely require attention.
  * **Yellow**: Potentially important problems that still warrant review.
  * **Grey**: Less severe or informational issues.

### Minor Issues

![](/_fern-img/90f44474a73f3ed84de0e535eaa05be11d4141e1609783374c63bbd0a406f1fd.webp)

Sometimes the agent produces issues that are determined to be less relevant to the current extraction. These are classified as "Minor Issues" and are:

* Down-weighted in scoring
* Hidden by default in the UI

You can view these by activating the minor issues toggle.

> **Note**
>
> Minor issues are usually precautionary in nature and may not reflect actual errors in the extraction.

### Issues for Arrays

Arrays have a hierarchical issue structure:

* **Common Issues**: Displayed at the top-level tooltip for the array. These are issues that apply to multiple items in the array.
  ![](/_fern-img/8a5b60cf0a8b6566c8fbfe455e420dee6f1e13f142516cf46fe6d31ad3eb8b59.webp)

* **Item-Specific Issues**: Targeted issues specific to a single item in the array. These are listed under the item-level review tooltip.
  * Item level issues are considered to be property level if they have specific properties affected

## Steering the Review Agent

You can guide the Review Agent's behavior to better suit your specific needs.
![](/_fern-img/418e848223ffe0f16d2c05fff1444a58f4ed1a7af73b2a868fd960ba4070b356.webp)

### Review Agent Instructions

You can add high level instruction to the Review Agent to teach it what to look for or what to ignore. This is useful for directing overall behavior of the review agent.

### Review Agent Rules

The review agent also supports structured rules to help drive the scoring of more specific edge cases you may find in your evaluation set. For issues that always require review or rejection of an extraction, you can set a rule to be "critical". These rules, if broken, will always set the score of the offending element to 1.

### Schema Improvements

The Review Agent picks up on ambiguities and issues that often trace back to the extraction schema. Sometimes, clarifying field descriptions or tightening the schema definition is more effective at driving correct Review Agent behavior than adding explicit rules.

# Review Agent API

## Enabling Review Agent

Review Agent can be enabled for **JSON Schema extractors** via `advancedOptions.reviewAgent.enabled`.

```json
{
  "type": "EXTRACT",
  "schema": { "type": "object", "properties": {} },
  "advancedOptions": {
    "reviewAgent": { "enabled": true }
  }
}
```

## Response Structure

The Review Agent adds two main fields to the metadata for each field, array item, and property:

* `reviewAgentScore`: An integer score from 1-5 indicating confidence (or `null` when Review Agent is enabled but does not return a score for a field).
* `insights`: An array of insights. Review Agent contributes `issue` and `review_summary` insights, and these may coexist alongside `reasoning` insights (if model reasoning insights are enabled).

### Metadata Object

```json
{
  "value": {
    "invoice_number": "INV-123",
    "line_items": [
      { "description": "Widget A", "amount": 100 },
      { "description": "Widget B", "amount": 150 }
    ]
  },
  "metadata": {
    "invoice_number": {
      "logprobsConfidence": null,
      "reviewAgentScore": 5,
      "insights": [
        {
          "type": "review_summary",
          "content": "No issues detected."
        }
      ]
    },
    "line_items": {
      "logprobsConfidence": null,
      "reviewAgentScore": 4,
      "insights": [
        {
          "type": "review_summary",
          "content": "Most items look consistent; some fields have potential issues."
        }
      ]
    },
    "line_items[0]": {
      "logprobsConfidence": null,
      "reviewAgentScore": 4
    },
    "line_items[0].amount": {
      "logprobsConfidence": null,
      "reviewAgentScore": 3,
      "insights": [
        {
          "type": "issue",
          "content": "Currency symbol is ambiguous."
        }
      ]
    },
    "line_items[1]": {
      "logprobsConfidence": null,
      "reviewAgentScore": 4,
      "insights": [
        {
          "type": "issue",
          "content": "Potential duplicate items detected in the list."
        }
      ]
    },
    "line_items[1].amount": {
      "logprobsConfidence": null,
      "reviewAgentScore": 4
    }
  }
}
```

## Field Descriptions

### `reviewAgentScore`

A generic confidence score ranging from 1 to 5. When Review Agent is enabled but no score is returned for a field, this value may be `null`.

* **5**: High confidence. No issues detected.
* **4**: Good confidence. Minor observations or precautionary notes.
* **3**: Moderate confidence. Some uncertainty or potential issues found.
* **2**: Low confidence. Likely contains errors or significant ambiguities.
* **1**: Very low confidence. Critical issues detected.

### `insights`

A list of insights. Each insight has a `type` and `content`.

* **`type: "issue"`**: A specific problem identified by Review Agent.
* **`type: "review_summary"`**: A general summary of the review findings from Review Agent for that field.
* **`type: "reasoning"`**: Model reasoning about the extraction decision (controlled by model reasoning insights, not Review Agent).

## Accessing Review Data

You can access Review Agent data programmatically by traversing the `metadata` object using standard path notation.

### Scalar Fields

For a non-array field like `invoice_number`, access the metadata directly by key.

```javascript
// reviewAgentScore may be null even when Review Agent is enabled
const score = response.metadata.invoice_number.reviewAgentScore ?? 5;
const insights = response.metadata.invoice_number.insights ?? [];
const issues = insights.filter((i) => i.type === "issue");
```

### Arrays

Arrays have metadata at up to three levels. Note that issues that affect specific items/properties will be distributed to the corresponding item/property keys.

1. **Array Level**: `response.metadata.line_items`
   * Contains the overall `reviewAgentScore` for the array.
   * `insights` may contain the `review_summary` for the array, and may rarely include issues if an issue applies to the array as a whole.
2. **Item Level**: `response.metadata['line_items[0]']`
   * Contains the score for the specific item (or `null` if no score was returned for that item).
   * Includes item-specific issues **and** any common issues that do not affect a specific property in the item. Issues at this level apply to the entire item
3. **Property Level**: `response.metadata['line_items[0].amount']`
   * Contains the score for the specific property (or `null` if no score was returned for that property).
   * Includes item-specific issues and common issues that apply to this specific property

## Code Examples

### Collecting All Issues for an Array

To gather all issues across an entire array (including item-level and property-level issues), iterate through all metadata keys that match the array pattern.

```typescript
interface Insight {
  type: "reasoning" | "issue" | "review_summary";
  content: string;
}

interface FieldMetadata {
  reviewAgentScore?: number | null;
  insights?: Insight[];
  logprobsConfidence: number | null;
}

type Metadata = Record<string, FieldMetadata>;

function collectAllArrayIssues(
  metadata: Metadata,
  arrayName: string
): Insight[] {
  const issues: Insight[] = [];
  const pattern = new RegExp(`^${arrayName}\\[\\d+\\]`);

  for (const [key, fieldMeta] of Object.entries(metadata)) {
    // Match item-level (line_items[0]) and property-level (line_items[0].amount)
    if (pattern.test(key) && fieldMeta.insights) {
      const fieldIssues = fieldMeta.insights.filter((i) => i.type === "issue");
      issues.push(...fieldIssues);
    }
  }

  return issues;
}

// Usage
const allLineItemIssues = collectAllArrayIssues(response.metadata, "line_items");
console.log(`Found ${allLineItemIssues.length} issues in line_items`);
```

```python
from typing import TypedDict, Literal

class Insight(TypedDict):
    type: Literal["reasoning", "issue", "review_summary"]
    content: str

def collect_all_array_issues(metadata: dict, array_name: str) -> list[Insight]:
    import re
    issues = []
    pattern = re.compile(rf"^{re.escape(array_name)}\[\d+\]")

    for key, field_meta in metadata.items():
        # Match item-level (line_items[0]) and property-level (line_items[0].amount)
        if pattern.match(key) and "insights" in field_meta:
            field_issues = [i for i in field_meta["insights"] if i["type"] == "issue"]
            issues.extend(field_issues)

    return issues

# Usage
all_line_item_issues = collect_all_array_issues(response["metadata"], "line_items")
print(f"Found {len(all_line_item_issues)} issues in line_items")
```

### Collecting Issues for a Specific Array Item

To collect all issues for a single array item (including its nested properties), filter metadata keys by the specific item index.

```typescript
function collectItemIssues(
  metadata: Metadata,
  arrayName: string,
  itemIndex: number
): Insight[] {
  const issues: Insight[] = [];
  const itemPrefix = `${arrayName}[${itemIndex}]`;

  for (const [key, fieldMeta] of Object.entries(metadata)) {
    // Match exact item (line_items[0]) or its properties (line_items[0].amount)
    if (key === itemPrefix || key.startsWith(`${itemPrefix}.`)) {
      if (fieldMeta.insights) {
        const fieldIssues = fieldMeta.insights.filter((i) => i.type === "issue");
        issues.push(...fieldIssues);
      }
    }
  }

  return issues;
}

// Usage: Get issues for the first line item
const firstItemIssues = collectItemIssues(response.metadata, "line_items", 0);

// Iterate through all items
const lineItems = response.value.line_items;
for (let i = 0; i < lineItems.length; i++) {
  const itemIssues = collectItemIssues(response.metadata, "line_items", i);
  if (itemIssues.length > 0) {
    console.log(`Item ${i} has ${itemIssues.length} issues:`);
    itemIssues.forEach((issue) => console.log(`  - ${issue.content}`));
  }
}
```

```python
def collect_item_issues(
    metadata: dict, array_name: str, item_index: int
) -> list[Insight]:
    issues = []
    item_prefix = f"{array_name}[{item_index}]"

    for key, field_meta in metadata.items():
        # Match exact item (line_items[0]) or its properties (line_items[0].amount)
        if key == item_prefix or key.startswith(f"{item_prefix}."):
            if "insights" in field_meta:
                field_issues = [
                    i for i in field_meta["insights"] if i["type"] == "issue"
                ]
                issues.extend(field_issues)

    return issues

# Usage: Get issues for the first line item
first_item_issues = collect_item_issues(response["metadata"], "line_items", 0)

# Iterate through all items
line_items = response["value"]["line_items"]
for i in range(len(line_items)):
    item_issues = collect_item_issues(response["metadata"], "line_items", i)
    if item_issues:
        print(f"Item {i} has {len(item_issues)} issues:")
        for issue in item_issues:
            print(f"  - {issue['content']}")
```

### Filtering Items by Score Threshold

A common pattern is to identify array items that need human review based on their score.

```typescript
interface ItemWithIssues {
  index: number;
  item: Record<string, unknown>;
  score: number;
  issues: Insight[];
}

function getItemsNeedingReview(
  response: { value: Record<string, unknown[]>; metadata: Metadata },
  arrayName: string,
  scoreThreshold: number = 3
): ItemWithIssues[] {
  const items = response.value[arrayName] as Record<string, unknown>[];
  const needsReview: ItemWithIssues[] = [];

  for (let i = 0; i < items.length; i++) {
    const itemKey = `${arrayName}[${i}]`;
    const itemMeta = response.metadata[itemKey];
    const score = itemMeta?.reviewAgentScore ?? 5;

    if (score <= scoreThreshold) {
      needsReview.push({
        index: i,
        item: items[i],
        score,
        issues: collectItemIssues(response.metadata, arrayName, i),
      });
    }
  }

  return needsReview;
}

// Usage
const itemsToReview = getItemsNeedingReview(response, "line_items", 3);
console.log(`${itemsToReview.length} items need human review`);
```

```python
from dataclasses import dataclass
from typing import Any

@dataclass
class ItemWithIssues:
    index: int
    item: dict[str, Any]
    score: int
    issues: list[Insight]

def get_items_needing_review(
    response: dict, array_name: str, score_threshold: int = 3
) -> list[ItemWithIssues]:
    items = response["value"][array_name]
    needs_review = []

    for i, item in enumerate(items):
        item_key = f"{array_name}[{i}]"
        item_meta = response["metadata"].get(item_key, {})
        score = item_meta.get("reviewAgentScore", 5)

        if score <= score_threshold:
            needs_review.append(
                ItemWithIssues(
                    index=i,
                    item=item,
                    score=score,
                    issues=collect_item_issues(response["metadata"], array_name, i),
                )
            )

    return needs_review

# Usage
items_to_review = get_items_needing_review(response, "line_items", 3)
print(f"{len(items_to_review)} items need human review")
```

## Using Scores in Workflows

You can use `reviewAgentScore` in conditional steps to route documents based on quality.

In `extend-web`, workflow conditions resolve `{{ ... }}` bindings using a simple dot-path lookup (e.g. `{{myStep.output.value.field}}`). This works well for scalar fields and array-level metadata keys, but **does not support** bracket/quoted access for metadata keys like `line_items[0]` or `line_items[0].amount`.

If you need coarse routing based on overall extraction quality, the extraction step execution context also exposes summary variables like `minReviewAgentScore`, `avgReviewAgentScore`, and `numFieldsFlaggedForReview`.

```javascript
// Route to human review if the invoice number score is low
{{extractionStep.output.metadata.invoice_number.reviewAgentScore}} <= 3

// Route based on an array field's overall score (array-level metadata key)
{{extractionStep.output.metadata.line_items.reviewAgentScore}} < 4

// Route based on the worst (minimum) Review Agent score across scalar fields
{{extractionStep.minReviewAgentScore}} <= 3

// Route if any fields were flagged for manual review
{{extractionStep.numFieldsFlaggedForReview}} > 0
```