How AI agents read PDFs: four patterns that actually work

An AI agent can’t usefully “open” a PDF — it needs the document turned into tokens, and how that happens decides the agent’s cost, accuracy, and autonomy. Four patterns cover real deployments: attach the PDF directly to the model, call a conversion API as a tool, expose conversion through a tool protocol like MCP, or run a local extraction library. Most production agents end up on pattern 2 or 3 for one reason: the agent can do it by itself, cheaply, mid-task.

Pattern 1: Attach the PDF to the model

Every major provider accepts PDF attachments and handles them with vision + extraction under the hood. Zero integration work, and it’s the right call for one-off documents in a chat.

The failure modes appear at agent scale: page-as-image processing costs 1,500–3,000 tokens per page (the math), the extraction is non-deterministic between runs, and the document must be re-attached — re-paid — on every conversation that needs it.

Pattern 2: A conversion API as a tool

Give the agent a convert_pdf tool that posts the file to a conversion endpoint and returns Markdown. The agent reads the Markdown as ordinary context — cheap, structured, cacheable.

def convert_pdf(path: str) -> str:
    """Tool: convert a text-layer PDF to Markdown."""
    import requests
    with open(path, "rb") as f:
        r = requests.post("https://api.marklipi.com/convert", files={"file": f}, timeout=60)
    if r.status_code == 422:
        return "ERROR: scanned PDF — no text layer. Use an OCR tool."
    r.raise_for_status()
    return r.json()["markdown"]

Two properties matter for agent use specifically. Keyless access: an agent can use marklipi without a human first creating an account and provisioning a secret — the tool works out of the box, within free-tier limits (10/min, 100/day). Legible errors: the 422-on-scans behavior gives the agent a decision point it can reason about (“this is a scan, I should say so or find OCR”) instead of silently degraded text.

Pattern 3: Conversion via MCP or a tool registry

Same conversion, different packaging: expose it as an MCP server (or equivalent tool-protocol wrapper) so any MCP-capable agent — Claude Desktop, IDE agents, custom stacks — gets PDF reading by configuration rather than code. The agent-facing contract is identical to pattern 2; what changes is distribution: one server definition serves every agent in the org, with the tool description teaching the model when (and when not) to use it.

Pattern 4: A local extraction library

Bundle pypdf, pdfplumber, or PyMuPDF into the agent’s runtime and extract in-process. Right when files can’t leave the machine, or when the agent needs geometry (bounding boxes, cell coordinates) rather than content. Costs: you own the dependency and its quirks (the Python options compared), and agents that write their own extraction code on the fly tend to produce the scrambled-table failure mode that poisons everything downstream.

Choosing, quickly

The scan column is the routing insight: agents do best with a cheap deterministic default plus an explicit OCR escalation path, rather than paying vision prices for every document because some documents are scans. (Text layer vs OCR, and how to route.)

FAQ

Can ChatGPT or Claude read PDFs without any of this?

Yes — attachments work in the chat products. These patterns are for built agents: pipelines and autonomous systems where per-page vision pricing, non-determinism, and manual re-attachment don’t scale.

What should the tool return when conversion fails?

A short, actionable string the model can reason about (“scanned PDF — no text layer; use OCR”) beats an exception. Agents recover well from legible errors and badly from stack traces.

How do agents handle PDFs behind URLs?

The agent fetches the bytes (its HTTP tool), then converts via its conversion tool. Upload-based conversion keeps private documents private — no third party fetching your URLs.