# How AI agents read PDFs: four patterns that actually work

> The four working patterns for giving an AI agent PDF-reading ability — direct file input, conversion APIs, MCP-style tools, and local libraries — with the failure modes of each and how to choose.

Published: 2026-07-30 · Source: https://marklipi.com/blog/how-ai-agents-read-pdfs


An AI agent can't usefully "open" a PDF — it needs the document turned into tokens, and *how* that happens decides the agent's cost, accuracy, and autonomy. Four patterns cover real deployments: attach the PDF directly to the model, call a conversion API as a tool, expose conversion through a tool protocol like MCP, or run a local extraction library. Most production agents end up on pattern 2 or 3 for one reason: the agent can do it **by itself, cheaply, mid-task**.

## Pattern 1: Attach the PDF to the model

Every major provider accepts PDF attachments and handles them with vision + extraction under the hood. Zero integration work, and it's the right call for one-off documents in a chat.

The failure modes appear at agent scale: page-as-image processing costs 1,500–3,000 tokens per page ([the math](/blog/markdown-vs-pdf-llm-token-costs)), the extraction is non-deterministic between runs, and the document must be re-attached — re-paid — on every conversation that needs it.

## Pattern 2: A conversion API as a tool

Give the agent a `convert_pdf` tool that posts the file to a conversion endpoint and returns Markdown. The agent reads the Markdown as ordinary context — cheap, structured, cacheable.

```python
def convert_pdf(path: str) -> str:
    """Tool: convert a text-layer PDF to Markdown."""
    import requests
    with open(path, "rb") as f:
        r = requests.post("https://api.marklipi.com/convert", files={"file": f}, timeout=60)
    if r.status_code == 422:
        return "ERROR: scanned PDF — no text layer. Use an OCR tool."
    r.raise_for_status()
    return r.json()["markdown"]
```

Two properties matter for agent use specifically. **Keyless access**: an agent can use marklipi without a human first creating an account and provisioning a secret — the tool works out of the box, within free-tier limits (10/min, 100/day). **Legible errors**: the `422`-on-scans behavior gives the agent a decision point it can reason about ("this is a scan, I should say so or find OCR") instead of silently degraded text.

## Pattern 3: Conversion via MCP or a tool registry

Same conversion, different packaging: expose it as an MCP server (or equivalent tool-protocol wrapper) so any MCP-capable agent — Claude Desktop, IDE agents, custom stacks — gets PDF reading by configuration rather than code. The agent-facing contract is identical to pattern 2; what changes is distribution: one server definition serves every agent in the org, with the tool description teaching the model when (and when not) to use it.

## Pattern 4: A local extraction library

Bundle pypdf, pdfplumber, or PyMuPDF into the agent's runtime and extract in-process. Right when files can't leave the machine, or when the agent needs geometry (bounding boxes, cell coordinates) rather than content. Costs: you own the dependency and its quirks ([the Python options compared](/blog/pdfplumber-vs-pymupdf-vs-pypdf)), and agents that write their own extraction code on the fly tend to produce the scrambled-table failure mode that poisons everything downstream.

## Choosing, quickly

| | Direct attach | API tool | MCP tool | Local library |
| --- | --- | --- | --- | --- |
| Setup | None | ~10 lines | Config | Dependency mgmt |
| Cost/page | High (vision) | ~Free tier | ~Free tier | Compute only |
| Deterministic | No | Yes | Yes | Yes |
| Handles scans | Yes | No (clean error) | No (clean error) | No (unless OCR added) |
| Agent autonomy | Needs file re-sent per call | Full | Full | Full |

The scan column is the routing insight: agents do best with a **cheap deterministic default plus an explicit OCR escalation path**, rather than paying vision prices for every document because some documents are scans. ([Text layer vs OCR, and how to route.](/blog/pdf-text-layer-vs-ocr))

## FAQ

### Can ChatGPT or Claude read PDFs without any of this?

Yes — attachments work in the chat products. These patterns are for *built* agents: pipelines and autonomous systems where per-page vision pricing, non-determinism, and manual re-attachment don't scale.

### What should the tool return when conversion fails?

A short, actionable string the model can reason about ("scanned PDF — no text layer; use OCR") beats an exception. Agents recover well from legible errors and badly from stack traces.

### How do agents handle PDFs behind URLs?

The agent fetches the bytes (its HTTP tool), then converts via its conversion tool. Upload-based conversion keeps private documents private — no third party fetching your URLs.