Cutting the cost of reading customer documents.
Role: Member of Technical Staff, mentoring and consulting Serhii Shchoholiev · Status: switching to the new approach in production
The problem
Extraction was expensive
The expensive part was document extraction: pulling out fields and their values, classifying each document, and working out which entity a document belongs to. Ingesting a customer’s documents, or redoing extraction, could run upwards of $10,000.
Who did what
Experiments and baselines
I mentored and consulted Serhii throughout this work. He built the test set (100 financial documents, Excel and PDF) and the OpenAI Decisions API experiments. I built the baselines to compare them against: approaches using embeddings, GLiNER models, and combinations of the two.
What won
Iterative choice
The winner was Serhii’s “iterative choice” approach. The first request asks which document type fits and whether the file is one document or a bundle of reports. Single documents stop there, and bundles keep picking from the remaining types until nothing else applies, up to five. It beat the agent on accuracy (98.1% vs. 93.5% F1) at about 1/7 of the cost and with 28× lower p90 latency.
My baselines
A useful floor
My baselines set a useful floor. Embedding-based approaches never got past about 60% correct classification, GLiNER models got close to 70%, and the combinations landed in between. That made it clear the much smaller models weren’t ready to replace the agent for classification on their own, while the Decisions API approach was both cheaper and more accurate.
Next
Smaller models for extraction
On the extraction side, I explored much smaller models like GLiNER2.5 that can pull out fields with far fewer parameters. The plan was to fine-tune one on Hypha’s production documents. That work is on hold, so there are no results yet.
Where it is today
About 60× cheaper
Hypha is now switching classification over to the new approach. The production estimate drops from about $6 to about $0.10 per 100 documents, roughly 60× cheaper, and even lower than the $0.89 from the original test.
What I learned
Baselines, always
I learned to keep up with new models as they come out, because the right tool for a step can change fast. Baselines are always necessary; that was true in my Ph.D. and it was true here. And, as one of my professors liked to say, almost any task can be turned into a classification task. The trick is in how you frame it.