# 7 ways to cut LLM token costs in document-heavy pipelines

> Practical, ranked techniques for reducing LLM spend when your pipeline processes documents: format conversion, caching, routing, context pruning, and the arithmetic to prioritize them.

Published: 2026-07-28 · Source: https://marklipi.com/blog/cut-llm-token-costs-document-pipelines


In document-heavy LLM pipelines — agents reading statements, RAG over reports, extraction from contracts — input tokens dominate the bill, and most of those tokens are the *document representation*, not your prompt. That makes representation the first lever to pull: converting PDFs to Markdown before the model sees them typically cuts document tokens 60–90%, and the remaining techniques compound on top.

Ranked by impact-per-effort:

## 1. Feed Markdown, not raw PDFs

Sending a PDF to a vision-capable model costs on the order of 1,500–3,000 tokens *per page* as pixels; the same page as Markdown is usually a few hundred. One keyless call does the conversion (`curl -s https://api.marklipi.com/convert -F "file=@doc.pdf"`). The [full arithmetic is here](/blog/markdown-vs-pdf-llm-token-costs) — for most pipelines this is the single biggest cut, and it also *improves* accuracy on tables.

## 2. Convert once, cache the Markdown

Agents re-read documents constantly — every retry, every follow-up. Re-converting (or worse, re-sending the PDF) pays full price each time. Key the converted Markdown by file hash and reuse it; deterministic extraction means the cache never goes stale for the same bytes.

## 3. Use prompt caching for repeated context

Every major provider now offers prompt caching with a large discount on cached input tokens. Structure prompts so the stable part — system prompt, document content — is a cacheable prefix and the variable part (the question) comes last. For multi-turn analysis of one document, this multiplies savings from #1.

## 4. Route documents to the cheapest capable model

Classification, field extraction, and summarization of clean Markdown rarely need your frontier model. A two-tier design — small model by default, escalate on low confidence — cuts cost on the 80% of documents that are routine. Clean input makes small models viable: much of what pushes teams to big models is compensating for messy extraction.

## 5. Retrieve sections, not documents

Stuffing a whole report into context because "the model might need it" bills you for every token every call. Chunk on Markdown headings and retrieve the sections relevant to the question — [the chunking guide](/blog/parse-pdfs-for-rag-chunking-guide) covers the mechanics. Long context windows make whole-document stuffing *possible*; they don't make it *economical*.

## 6. Strip what models don't need

Page furniture (repeated headers/footers, page numbers), boilerplate legal footers, decorative whitespace — all tokens, no signal. Good converters drop most furniture during extraction; a light post-pass on your corpus's known boilerplate finishes the job.

## 7. Batch and deduplicate

Batch APIs price lower for non-urgent work — overnight document backfills belong there. And corpora are full of near-duplicates (the same contract template 400 times): hash-based dedupe before processing, then process the template once and the deltas separately.

## The arithmetic that sets priorities

A worked example — 10,000 documents/month, ~20 pages each, one analysis pass:

| Setup | Tokens/doc | Monthly input tokens |
| --- | --- | --- |
| Raw PDF to vision model | ~40,000 | 400M |
| Markdown (technique #1) | ~8,000 | 80M |
| + caching & retrieval (#2, #3, #5) | ~3,000 effective | 30M |

The order matters: #1 is a one-line change that removes 80% of the bill, which is why representation beats clever prompting as the first optimization. Price the difference at your model's input rate and it funds the rest of the work.

## FAQ

### What's the single fastest win?

Convert PDFs to Markdown before the model sees them ([how](/blog/convert-pdf-to-markdown-python-node-curl)). One pipeline step, 60–90% fewer document tokens, better table accuracy.

### Do these techniques hurt quality?

#1 and #6 typically *improve* it (cleaner input), #3 and #7 are quality-neutral, and #4 and #5 trade a small risk (wrong routing, missed retrieval) for large savings — mitigate with confidence-based escalation.

### Does this apply with long-context models?

More than ever: long contexts make it easy to spend 200k tokens per call. The cheaper your document representation, the more actual documents fit in the window — and the less you pay for each.