"PDF structure, built for speed."

What is pdf-inspector Overview

pdf-inspector is a fast, open-source Rust library by Firecrawl focused on PDF classification, text extraction, and Markdown conversion.

Its mission is precise: be the first gate of any hybrid OCR pipeline. OCR is slow and expensive, yet about 54% of PDFs are native text and never need it. pdf-inspector classifies a document as text-based or scanned within ~10–50ms; text-based ones complete extraction and conversion locally in under 200ms — only the pages that truly need OCR get routed there. It is the underlying engine of Firecrawl's official PDF processing chain.

It is not another wrapper: the default build is pure Rust with no ML models and no external services — its only PDF parsing dependency is lopdf. The whole extraction is one purely local parse.

Core features Core Features

• Smart classification: TextBased / Scanned / ImageBased / Mixed in ~10–50ms, with a 0–1 confidence score and per-page OCR routing signals.
• Region-aware extraction: every text item carries X/Y coordinates, font, and size; extract from bounding boxes with per-region needsOcr quality flags.
• Layout-aware: multi-column and newspaper layouts reconstructed into correct reading order; RTL support.
• Robust decoding: CID / Type0 fonts decoded via ToUnicode CMaps across UTF-16BE / UTF-8 / Latin-1; broken encodings flagged so callers can fall back to OCR.
• Markdown conversion: headings (H1–H4 scaled by font size), bulleted/numbered/alphabetized lists, code blocks (monospace detection), tables, bold/italic, links, page markers.
• Dual-mode table detection: rectangle drawing ops + text-alignment heuristics recover financial tables and cross-page continuations.
• Single document load: parsed once; detection and extraction share the same parse.
• Selective OCR (optional): native packages can route only rejected pages to OCR while everything else stays on the fast path, with per-page provenance (native / ocr / fused).
• Browser WASM: CMaps embedded so CJK font decoding never touches a filesystem; single-threaded build needs no cross-origin isolation.
• Well tested: snapshot tests, mutation testing, per-module cargo-fuzz targets.

Architecture Architecture

The philosophy in one sentence: one parse, reused everywhere.

PDF file / bytes
        │
        ▼
  detector —— classification (~10–50ms)
  TextBased / Scanned / ImageBased / Mixed + confidence
        │
        ▼
  extractor —— content-stream text pulls
  (position, font, size via ToUnicode CMap)
        │
        ▼
  layout —— lines / paragraphs / reading order
  (multi-column reassembly, RTL, ligatures)
        │
        ▼
  tables —— drawn rects + alignment heuristics
        │
        ▼
  markdown —— semantic GFM output

Key building blocks:

Component	Role
Rust	Core language, compiled to native / WebAssembly
lopdf	The only PDF parsing dependency
detector.rs	Fast classification (sampled pixel statistics)
extractor/	Region-aware extraction pipeline
tounicode.rs	ToUnicode CMap parsing
napi/ · wasm/	Node.js & browser bindings

Note: native packages embed no OCR models, PDFium, or ONNX Runtime by default — clean processing requests never load or download these components. They only matter when you explicitly opt into OCR and pages actually get routed there.

The four PDF types Classification

Type	Meaning	Recommended handling
TextBased	Native text living in the content stream	Extract locally — fast and free
Scanned	Whole-page images	Needs OCR
ImageBased	Image-only (e.g. design exports)	Needs OCR
Mixed	Some pages text, some images	Route per page; only OCR what's needed

Results include a confidence score (0.0–1.0) plus an exact list of pages needing OCR, so you control downstream costs precisely.

Benchmark Benchmark

Official comparison on the opendataloader-bench corpus (200 real PDFs), local engines without model-based PDF parsing, OCR disabled (scores 0–1, higher is better):

Engine	Overall	Reading order	Tables (TEDS)	Headings	Speed
pdf-inspector	0.875	0.915	0.814	0.788	0.470s
liteparse	0.873	0.913	0.693	0.811	0.750s
opendataloader	0.831	0.902	0.489	0.739	2.569s
pymupdf4llm	0.735	0.886	0.401	0.424	17.117s
markitdown	0.589	0.844	0.273	0.000	16.165s

Bottom line: most accurate overall, strongest tables, fastest (headings second only to liteparse). See the benchmark page.

License License

pdf-inspector is released under the MIT License — free to use, modify, and commercialize. Source at github.com/firecrawl/pdf-inspector; installable via crates.io (Rust/CLI), npm (Node/WASM), and PyPI (Python).

Try it now Try it now

• Nothing to install? Drop a PDF into the demo on the home page — parsed locally right in your browser.
• Want it in your project? Pick your language on the install page.
• Prefer the full tour? Head to the quick start guide.

In one sentence

pdf-inspector = classify first + a five-stage pipeline + one clean Markdown output. Native-text PDFs never pay OCR prices — fully local, MIT, LLM-friendly.