In document-heavy LLM pipelines — agents reading statements, RAG over reports, extraction from contracts — input tokens dominate the bill, and most of those tokens are the document representation, not your prompt. That makes representation the first lever to pull: converting PDFs to Markdown before the model sees them typically cuts document tokens 60–90%, and the remaining techniques compound on top.
Ranked by impact-per-effort:
1. Feed Markdown, not raw PDFs
Sending a PDF to a vision-capable model costs on the order of 1,500–3,000 tokens per page as pixels; the same page as Markdown is usually a few hundred. One keyless call does the conversion (curl -s https://api.marklipi.com/convert -F "file=@doc.pdf"). The full arithmetic is here — for most pipelines this is the single biggest cut, and it also improves accuracy on tables.
2. Convert once, cache the Markdown
Agents re-read documents constantly — every retry, every follow-up. Re-converting (or worse, re-sending the PDF) pays full price each time. Key the converted Markdown by file hash and reuse it; deterministic extraction means the cache never goes stale for the same bytes.
3. Use prompt caching for repeated context
Every major provider now offers prompt caching with a large discount on cached input tokens. Structure prompts so the stable part — system prompt, document content — is a cacheable prefix and the variable part (the question) comes last. For multi-turn analysis of one document, this multiplies savings from #1.
4. Route documents to the cheapest capable model
Classification, field extraction, and summarization of clean Markdown rarely need your frontier model. A two-tier design — small model by default, escalate on low confidence — cuts cost on the 80% of documents that are routine. Clean input makes small models viable: much of what pushes teams to big models is compensating for messy extraction.
5. Retrieve sections, not documents
Stuffing a whole report into context because “the model might need it” bills you for every token every call. Chunk on Markdown headings and retrieve the sections relevant to the question — the chunking guide covers the mechanics. Long context windows make whole-document stuffing possible; they don’t make it economical.
6. Strip what models don’t need
Page furniture (repeated headers/footers, page numbers), boilerplate legal footers, decorative whitespace — all tokens, no signal. Good converters drop most furniture during extraction; a light post-pass on your corpus’s known boilerplate finishes the job.
7. Batch and deduplicate
Batch APIs price lower for non-urgent work — overnight document backfills belong there. And corpora are full of near-duplicates (the same contract template 400 times): hash-based dedupe before processing, then process the template once and the deltas separately.
The arithmetic that sets priorities
A worked example — 10,000 documents/month, ~20 pages each, one analysis pass:
| Setup | Tokens/doc | Monthly input tokens |
|---|---|---|
| Raw PDF to vision model | ~40,000 | 400M |
| Markdown (technique #1) | ~8,000 | 80M |
| + caching & retrieval (#2, #3, #5) | ~3,000 effective | 30M |
The order matters: #1 is a one-line change that removes 80% of the bill, which is why representation beats clever prompting as the first optimization. Price the difference at your model’s input rate and it funds the rest of the work.
FAQ
What’s the single fastest win?
Convert PDFs to Markdown before the model sees them (how). One pipeline step, 60–90% fewer document tokens, better table accuracy.
Do these techniques hurt quality?
#1 and #6 typically improve it (cleaner input), #3 and #7 are quality-neutral, and #4 and #5 trade a small risk (wrong routing, missed retrieval) for large savings — mitigate with confidence-based escalation.
Does this apply with long-context models?
More than ever: long contexts make it easy to spend 200k tokens per call. The cheaper your document representation, the more actual documents fit in the window — and the less you pay for each.