Convert Everything to Markdown for AI Agents — Part 4: anydoc vs markitdown vs /utils:pdf

Key Takeaways: No converter wins on all axes. anydoc (Firecrawl, Rust) converts a 10-page paper in 56 ms and was the only one of the two DOCX-capable tools that read our Word sample’s headings correctly — but it dropped the fi/ff ligatures in a LaTeX PDF, turning “efficient” into “ecient” with no warning. markitdown keeps text correct but emits no headings for PDFs, glues justified words together, and hallucinates Markdown tables out of two-column layouts. /utils:pdf (pymupdf4llm) produced the only usable structure on both PDFs, at the cost of being the slowest on the LaTeX paper. Across a follow-up corpus of 55 real PDFs — invoices, contracts, bank-statement specs, client specs, papers — the ligature bug hit 3 documents, all LaTeX-produced, while markitdown produced exactly 1 heading in the entire corpus. Rule: anydoc for Word, /utils:pdf for PDFs, markitdown for PowerPoint and spreadsheets — and always grep the output before trusting it.

Parts 1–3 of this series covered PDF, PowerPoint and Word one format at a time. Since then Firecrawl released anydoc, a Rust library that claims sub-5 ms median conversions across 14 formats, and our own internal debate about markitdown versus the /utils:pdf skill never actually got settled with numbers.

So we settled it with numbers.

The Setup

Three converters, one corpus, same machine:

Six inputs, chosen because each one breaks something different:

• paper.pdf — 10-page LaTeX conference paper, two-column, tables, formulas, embedded ligatures
• vi.pdf — 15-page Vietnamese journal paper, heavy diacritics, justified text
• vi.docx — the Word source of that same paper, numbered headings
• deck.pptx — 32-slide presentation with images and speaker notes
• sheet.xlsx — a spreadsheet with float columns
• scanned.pdf — the first two pages of paper.pdf rasterised at 150 DPI, i.e. no text layer at all

A second, larger corpus — 55 real PDFs — follows further down, to answer the question one document per format cannot: how often each failure actually occurs.

Times are the median of three runs (re-measured over seven for the LaTeX paper: 5–16% spread between fastest and slowest run), wall clock, including interpreter startup — which is a real cost when a converter runs once per document in a pipeline, not a benchmark artifact to be excluded.

Speed

anydoc’s headline claim holds on the LaTeX paper: 22× faster than markitdown, 44× faster than the pymupdf4llm-based skill. On the Vietnamese PDF the gap collapses to 1.4×, because that document is dominated by glyph-level work that no implementation language can skip.

A large slice of the Python cost is startup, not conversion — half of markitdown’s, a sixth of pymupdf4llm’s. Importing markitdown takes 658 ms on this machine and importing pymupdf4llm 380 ms (median of five), against ~12 ms for a bare interpreter and ~17 ms for a bare node. Subtract that and the actual conversion work is roughly 0.6 s for markitdown, 2.1 s for pymupdf4llm and 36 ms for anydoc. It matters which number you care about: a batch job that converts a thousand documents in one process pays the import once, while a per-document CLI call pays it every time.

Speed is the least interesting column, though. At these document sizes all three are fast enough for interactive use. What matters is what came out.

Correctness: the Ligature Bug

The LaTeX paper renders fi, ff and ffi as single ligature glyphs. anydoc extracts the glyph and drops the mapping:

markitdown : "this approach is effective" ... "depth-first search"
/utils:pdf : "this approach is effective" ... "depth-first search"
anydoc     : "this approach is eective"  ... "depth-rst search"

Whole-document, word-boundary counts of four sample words and their corrupted forms:

(markitdown’s totals run low because some occurrences are swallowed by the glued-word runs described in the next section — not because it recovered fewer ligatures.)

Twenty-four corrupted occurrences from four sample words alone. This is the failure mode we least want from a converter: it does not crash, it does not warn, and the output looks fine at a glance. An agent reading prot instead of profit reasons about a word that does not exist, and anything summarised from that text carries the corruption forward. How often does it happen? See the corpus run below: 3 documents out of 55, all of them LaTeX-produced academic papers, and none of the business documents. So it is a narrow risk with a wide blast radius — any PDF whose fonts encode fi/ff/ffi as ligature glyphs without a usable ToUnicode mapping is a candidate, and LaTeX output routinely is.

anydoc also mangled TeX en-dashes ([2, 6{8, 10] instead of [2, 6–8, 10]).

Correctness: markitdown’s Invented Tables

markitdown got the characters right and the structure badly wrong. On paper.pdf it emitted 135 table rows — from a document containing exactly two tables. pdfminer sees a two-column layout, decides the columns are table cells, and produces this:

| 2   | Problem |           | definition |     | and related |              | work |     |
| --- | ------- | --------- | ---------- | --- | ----------- | ------------ | ---- | --- |
| We  | first   | introduce | important  |     | preliminary | definitions. |      |     |

That is a section heading and a sentence, shredded into a fake table. It also glued justified text: 58 runs of 25+ characters with no space, like Highutilityitemsetminingisachallengingtask. Neither of the other two produced a single glued run.

And markitdown emitted zero headings on both test PDFs — flat text, the exact limitation we described in Part 1, still true two versions later. The corpus run below shows this is not a property of these two files.

Structure Scoreboard

Headings detected, and table rows emitted against reality:

The DOCX result is the surprise. The Word source has numbered headings styled as headings; markitdown flattened them into an ordered list (1. Giới thiệu), losing the level entirely, while anydoc produced ## I Giới thiệu. On this document anydoc is simply better — and a DOCX normally stores the characters themselves rather than font glyph codes, so the PDF font-mapping failure above does not arise (a document can still contain literal Unicode ligature characters, which is a different problem).

Vietnamese diacritics survived intact in all three. On this corpus, that was the one thing none of them got wrong.

At Scale: 55 Real PDFs

One document per format proves a bug exists; it says nothing about how often you will meet it. So we ran the same three tools over 55 deduplicated real PDFs pulled off this machine — supplier invoices, employment and tenancy contracts, an NDA, a SaaS agreement, the ISO 20022 camt.053 statement specification, client functional specs, factory worksheets, barcode sheets, CVs, and academic papers in English and Vietnamese. No document content is reproduced here; only aggregates.

Ligature loss is detected without a hand-written word list and without a dictionary: the reference vocabulary for each document is the combined output of the other two tools. Any tokenThe basic unit of text processed by an LLM. A token is roughly 4 characters or 0.75 words in English. LLMs process and generate text as sequences of tokens. Tokenization varies by model and language. missing from that vocabulary which lands back inside it when fi/fl/ff/ffi/ffl is re-inserted counts as a hit. It follows that the method only catches corruption that at least one other tool got right — a failure all three share would be invisible to it.

Three findings survive the jump from one document to 55.

The ligature bug is narrow but not rare. All 194 hits land in three academic papers — ospring for offspring (32×), dened for defined (25×), denition, prot, efcient, articial. We detected no ligature loss in any invoice, contract or specification in the corpus. If your pipeline eats business documents, this particular failure did not show up; if it eats papers or LaTeX-typeset specs, it did.

markitdown’s missing headings are not a quirk of our sample. One heading. In 55 documents. Every other PDF came back as flat text, exactly as in Part 1. Its 1,359 glued runs are five times anydoc’s and pdfskill’s combined, and those glued runs are also why its word count comes out lowest — words fused together stop being words.

anydoc’s speed advantage holds and grows. 32 ms median against ~1 s for both Python tools, and 23 s against 113 s and 189 s for the full corpus. Extrapolated to a thousand documents of the same mix, that is about 7 minutes against 34 and 57.

Scanned PDFs: One Honest Failure, Two Silent Ones

The corpus contained six genuinely image-only PDFs (factory worksheets, barcode sheets, a scanned statement walkthrough) alongside our rasterised test file. The pattern was identical on all seven:

anydoc     → exit 1: "unsupported input: PDF has no extractable text (Scanned, 2 pages): OCR is required"
markitdown → exit 0, 1 byte of output
/utils:pdf → exit 0, 1–971 bytes of output   (no tesseract installed)

anydoc refused all seven, every time, with a message naming the cause. That is the correct behaviour. markitdown’s default PDF path is text-extraction only — OCR requires opting into a separate backend such as Azure Document Intelligence — and it reports success anyway.

/utils:pdf does have an OCR fallback, and the empty output above is a missing dependency, not a missing featureAn individual measurable property or characteristic of the data used as input to a model. Feature engineering — selecting, transforming, and creating features — is a critical step in the ML pipeline.: it needs the system tesseract binary. We installed it and re-ran:

Quality on the rasterised paper: 99.6% word recall against the original text layer (272 of 273 words), correct paragraph reflow, and — as Part 1 predicted — a flat heading hierarchy. Both headings came back as ##, because once you are reading pixels the font-size signal that separates # from ### is gone.

The dependency gap is the real lesson. A skill that degrades to a 1-byte success when a binary is missing loses documents quietly. Install tesseract, or make the size assertion below non-optional.

If you run any of these unattended, assert on output size:

convert_or_die() {   # $1 = source document, $2 = markdown output
  [ "$(wc -c < "$2")" -gt 200 ] || { echo "conversion produced nothing: $1" >&2; exit 1; }
}

anydoc report.pdf > report.md
convert_or_die report.pdf report.md

PPTX and XLSX

markitdown won PowerPoint outright on our deck. It keeps slide order, marks each slide (<!-- Slide number: 12 -->), preserves speaker notes and references embedded images. anydoc emitted no slide boundaries and reordered shapes — the subtitle appeared before the title, because it walks shapes in XML order rather than reading order. On a deck, order is meaning.

Spreadsheets: both are flawed in opposite directions. markitdown leaks pandas artifacts (Unnamed: 8, NaN) but keeps sheet names and full precision. anydoc drops the sheet name, emits an empty header row, and rounds floats to four decimals — 0.124675 became 0.1247. Silent precision loss in a financial export is not a formatting nit.

Speed follows the same shape as the PDF numbers — on a single run anydoc converted the deck and the spreadsheet roughly 15–20× faster than markitdown — but on these two formats the faster tool is the one losing information.

The Rule We Ended Up With

anydoc handles ten more formats we did not test (RTF, ODT, EPUB, CSV and others); this benchmark says nothing about them either way.

So the internal debate resolves as “both, plus a third” — which is less satisfying than a single winner and more useful than one. The three tools are not competing implementations of the same thing; they are three different trade-offs between speed, text fidelity and structure recovery.

Verify, Then Trust

Every one of the failures above was found by grepping the output, not by reading release notes. For PDFs, three checks cost nothing and catch the three that bite hardest:

# 1. Did anything come out at all?
wc -c out.md

# 2. Ligature loss (LaTeX PDFs) — should be 0
grep -cE '\b(ecient|eective|prot|rst|denition|dicult|signicant)\b' out.md

# 3. Hallucinated tables — compare against the real table count
grep -c '^|' out.md

Other formats need their own checks: slide count and slide order for PPTX, heading count for DOCX, and a spot-check on decimal places for XLSX. The three greps above do not see any of those.

That is the same discipline Part 1 applied to footers and Part 2 applied to slide chrome. The converter is not the pipeline. The verification is.

Both corpora are our own and client documents, so we cannot ship them — but the method transfers, and the second half is the part worth copying: run the three tools over every PDF already sitting on your machine, then count the failures instead of inspecting them. The detector needs no word list, only a second and third tool to disagree with — treat that disagreement as a place to look, not as proof of who is right. It took an afternoon and it settled an argument that had been running for months.