pdf-inspector Benchmark - Five Engines Head to Head
How fast and how accurate is pdf-inspector, really? Rather than taking our word for it, here is the official head-to-head data on the opendataloader-bench corpus.
Methodology
- Sample: the opendataloader-bench corpus — 200 real PDFs
- Engines: pdf-inspector, liteparse, opendataloader, pymupdf4llm, markitdown (all local engines without model-based PDF parsing; OCR disabled)
- Dimensions: overall quality, reading order, table structure (TEDS), heading detection — scores 0–1, higher is better
- Speed: total time for a complete corpus run (Apple M4 Pro; median of five complete runs after an excluded warm-up)
Results
| Engine | Overall | Reading order | Tables (TEDS) | Headings | Speed |
|---|---|---|---|---|---|
| pdf-inspector | 0.875 | 0.915 | 0.814 | 0.788 | 0.470s |
| liteparse | 0.873 | 0.913 | 0.693 | 0.811 | 0.750s |
| opendataloader | 0.831 | 0.902 | 0.489 | 0.739 | 2.569s |
| pymupdf4llm | 0.735 | 0.886 | 0.401 | 0.424 | 17.117s |
| markitdown | 0.589 | 0.844 | 0.273 | 0.000 | 16.165s |
Data refreshed July 31, 2026. Full methodology and versions are in the repo README; raw timings and artifacts live in the results branch.
How to read this
Overall: 0.875, #1. It's neck-and-neck with liteparse (0.873) at the top, but the gap widens in the two dimensions below.
Tables: TEDS 0.814, leading by a wide margin. The runner-up sits at 0.693 and opendataloader at 0.489. The dual-mode design (drawn rectangles + alignment heuristics) clearly wins on financial tables and cross-page continuations — exactly where LLM pipelines break most often.
Speed: 0.470s for all 200 documents, fastest in the field. Roughly 37% faster than liteparse, over 5× faster than opendataloader, and more than 30× faster than pymupdf4llm or markitdown. In bulk ingestion or real-time parsing this is an experience-level difference.
Reading order: 0.915, also #1. Multi-column and newspaper layouts are reconstructed closest to natural human reading order — which directly determines how readable the resulting Markdown is.
Headings: 0.788, second only to liteparse (0.811). markitdown scores 0.000 — it essentially doesn't recover heading levels at all.
Why it is both fast and accurate
Speed isn't magic — it's architecture:
- Single load — the document is parsed once; detection, layout, tables, and Markdown share that parse
- Pure Rust — no ML models and no external services in the default build; lopdf is the only dependency
- Dual-mode table detection — drawing ops and alignment heuristics complement each other instead of betting on one
- Layout first — lines, paragraphs, and reading order are rebuilt before serialization rather than emitting content-stream order
Honest limitations
- No OCR in the benchmark: every engine ran with OCR off — only local parsing was compared. Scanned-document performance depends on each engine's OCR story.
- It's one benchmark: 200 documents are representative but not the whole universe; extreme layouts still deserve a real-world test.
- Timing is a median of full runs: single very large files take longer; typical end-to-end time for a text-based PDF stays under 200ms.
Want to run it yourself?
Methodology and versions are documented in the repo README; raw timings and artifacts are public on the opendataloader-bench results branch.
Want the head-to-head with markitdown?
Here it is — markitdown vs pdf-inspector: which one for PDF → Markdown?