“Convert PDF to JSON” usually means one of two different wishes: a structural dump (the document as nested blocks/paragraphs/coordinates) or extracted fields ({"invoice_number": …, "total": …}). For the first, JSON is the natural format. For the second, the robust pipeline in the LLM era is almost always PDF → Markdown → LLM → your JSON schema — because no converter knows your schema, but a model reading clean Markdown does.
The two meanings of “PDF to JSON”
Structural JSON is what layout tools emit: pages, blocks, lines, bounding boxes. You want it when position is meaning — form reconstruction, redaction, visual diffing. It’s verbose (often 10–50× the text’s size), and something still has to interpret it.
Semantic JSON — the fields your application needs — requires understanding, not just geometry: which of the seven numbers on an invoice is the total, and what the payment terms imply. That’s interpretation, and interpretation is now the LLM’s job.
Why Markdown is the better middle layer
A direct PDF-to-semantic-JSON converter has to hardcode assumptions about your documents. A two-step pipeline separates the mechanical from the intelligent:
- Mechanical, deterministic: extract the text layer to Markdown — exact characters, tables intact, 60–90% fewer tokens than the raw PDF.
- Intelligent, yours: prompt a model with the Markdown plus your schema and validation rules.
import requests, json
with open("invoice.pdf", "rb") as f:
md = requests.post("https://api.marklipi.com/convert",
files={"file": f}, timeout=60).json()["markdown"]
# Any LLM with structured output; schema is yours, not the converter's
fields = extract_with_llm(
md,
schema={"invoice_number": "str", "issue_date": "date",
"total": "decimal", "currency": "str", "line_items": "list"},
)
The division of labor is the point: extraction errors and interpretation errors stay separable. If a field is wrong, you can see whether the Markdown was wrong (it wasn’t, if the PDF had a text layer — extraction is deterministic) or the model misread it (fixable with prompt/schema changes). Direct PDF-to-JSON black-boxes fuse the two failure modes together.
When you genuinely want JSON at the extraction step
- Coordinates matter: redaction, highlighting, click-to-source UIs need bounding boxes — use a layout tool that emits structural JSON (several of these tools do).
- No LLM in the loop: fixed-format documents parsed with regex/positions at massive scale — structural JSON plus deterministic rules can beat model costs.
- Table-only extraction: if the entire job is “get this table,” Markdown-table → CSV/JSON is a one-liner, no model needed (table extraction guide).
The scan caveat, as always
Everything above assumes a text layer. Scanned PDFs make step 1 probabilistic (OCR), which changes how much you trust step 2’s output — route them explicitly rather than letting OCR noise masquerade as extracted fact. For agent pipelines, the 422-on-scan behavior of the conversion step is the routing signal (agent patterns here).
FAQ
What’s the best way to convert PDF to JSON?
Decide which JSON you mean. Document structure with coordinates → a layout-analysis tool. Application fields → convert to Markdown (one keyless call: curl -s https://api.marklipi.com/convert -F "file=@doc.pdf"), then extract with an LLM against your own schema.
Isn’t Markdown lossy compared to structural JSON?
It drops geometry (positions, fonts) and keeps semantics (headings, tables, order). For field extraction and RAG, that’s the right trade — the discarded geometry is exactly what inflates tokens without adding meaning.
Can I skip Markdown and prompt the LLM with the raw PDF?
You can — at vision-model token prices, with non-deterministic reads on the numbers you care about. Markdown-first is cheaper and reproducible; the token math quantifies it.