pdfParse

At a glance

• Outcome: Point it at a large body of PDFs and every extracted fact keeps a link back to exactly where it came from, so downstream retrieval and chat can cite instead of hallucinate.
• Status: Working prototype. Ingest, verify, revision supersession and a read-only query mode all exist and are tested. The job-run verbs are still placeholders, and a corpus service, web UI and MCP surface are out of scope on purpose, since those belong to Qurite.
• Role: Solo. This is the PDF extraction component of Qurite.
• Stack & libraries: Rust, as an engine library, a thin CLI and a contract crate versioned independently of the engine. The output is a portable vector dataset carrying the values, their locations, a manifest and the action log.
• Source: MIT-licensed and release-prepped, but not published yet.
• Limitations: Proven against one golden corpus, whose own reference answer is still partly unverified and which isn’t mine to redistribute, so the tests that need it skip in a public checkout. Broad-document generality is exactly what hasn’t been demonstrated.

The problem it’s for

Retrieval over documents usually launders away the one thing that makes an answer checkable. A chunk gets embedded, a model gets a similar chunk back, and the answer that comes out has no way to point at where it came from. For anything consequential, like a contract or a spec or a structural drawing set, that’s disqualifying.

Most retrieval tools also assume a PDF is a sequence of text chunks. A structural drawing set isn’t. So before extracting anything, the engine decides how a document is organized: as a hierarchy, where a section is a node in an outline, or spatially, where a section is a region on a page. That one decision changes what “section” means for everything downstream. The drawing set here has a perfectly good text layer and is still spatial, because reading order tells you nothing about which beam is which.

The trap the golden case is built around

The proving case is a structural drawing set: find, count and locate every W18X40 beam callout. It’s a good test because it has an obvious wrong answer sitting right next to the right one.

Every callout reads W18X40 (44) <1 3/4">, and the naive read is “44 beams.” It isn’t. The number in parentheses is a stud count, a property of each beam that happens to be the same on every one of them in this set, and the angle brackets hold the camber. The quantity you want is the number of callouts, and a correct highlight draws exactly one box per callout.

So the system had to name a unit. A callout instance is one occurrence of a member label with its own page and its own box, and the parsing rule is written down: the parenthesized number is a stud count and never a quantity. Most of the provenance machinery exists to keep that distinction from collapsing somewhere between the page and the answer.

The reference answer records 294 callouts across ten of the eleven pages, each with a box, pulled from the text layer.

It isn’t yet a hand-verified count of physical beams, and the fixture says so. No revision-cloud analysis has been done, so on a set merged through 22 change orders all 294 are provisionally treated as current when some may be superseded. A handful carry no camber in the text layer, and a handful of stud tokens disagreed with the design constant and got normalized and flagged. Those open items live in the fixture instead of being rounded off, because the whole point of the case is that the reference has to be as auditable as the output.

What an extraction’s identity actually is

An extraction isn’t addressed by where it came from alone. Its identity is a hash of the source, the page, the region quantized to a whole number, the kind of extraction, and the versions of the config and models that produced it.

The quantization is there because floating point isn’t byte-identical across languages, and the identity has to be recomputable by a consumer in another language that never links this engine.

The model never contributes geometry to that key. It picks the neighborhood, and deterministic code snaps the box to the actual content underneath. So an unchanged rerun updates in place instead of duplicating, and model jitter can’t mint a new fact. On a scanned page there’s nothing to snap to, so identity falls back to a grid cell, the record is stamped as AI-suggested, and it’s explicitly exempt from that guarantee.

How the LLM is kept on a leash

The model’s authority is bounded. Its output is grammar-constrained, so it can’t return something the pipeline can’t parse. Data and instructions are separate channels, so document contents can’t act as instructions. Personal information is masked before anything gets embedded. And model interactions are recorded so they can be replayed.

The append-only action log is the source of truth, not the extractions table. Whether a record is active, superseded or suppressed is computed by folding the log, so there’s no mutable status column for a race to corrupt. Undo is a forward event: reverting to an earlier state appends a record saying so instead of rewinding, so the history of why the state changed survives the reversal. The extraction and evidence tables are projections. Drop them, replay the log, and you get the same tables back without asking a model anything. That’s also how a crashed job resumes.

And there’s a question the system is required to refuse. “How many physical W18X40 beams are in the project?” is marked unsupported in the evaluation set. The expected result is not-enough-evidence, and the two answers a confident model would give, the callout count and the stud count, are written into the fixture as forbidden. A test enforces it. The reference has never established a verified beam count, so the eval refuses to let the answer exist. Knowing which questions the corpus can’t answer is the same problem as citing the ones it can.

What it costs

This isn’t free, and the trade is worth naming. Every model decision is written and flushed to disk before its effect is applied, which puts an ordering constraint and a durable write on the hot path. Retention keeps everything by default, so superseded rows are unindexed rather than deleted and the log grows with activity. Reclaiming space means an explicit compaction that is itself a logged event. Reads pay too, since nothing can trust the extractions table directly and has to rebuild it from history.

I took that trade because a crash, a retry or a reversal that can’t be explained afterward is worse than a slow write.

What the tests actually prove

Two tests split the work. One reads the actual PDF bytes and pulls all 294 designations out of the text layer, matching the reference page by page, including a callout rotated ninety degrees. The other never opens the PDF: it builds records from the reference, replays a recorded model session, folds everything into a projection, and asserts that running it again writes nothing new.

The first proves extraction. The second proves orchestration, idempotence and replay, and runs without a GPU.

The drawing set isn’t mine to redistribute, so the public release drops it. The tests that need it detect its absence and skip, and a private copy at the original path re-enables them.