The method
Five principles, every one of them learned by breaking something.
- 01
The evaluation set before the pipeline
30 stratified files, 8 critical fields, 240 cells annotated by hand, sample drawn deterministically so it can be replayed. Without it, “is this better?” is judged by eye on a handful of familiar queries. I did it afterwards. I’d do it first.
- 02
Cost is a feature, not a line on an invoice
Not every question deserves the same model. Classifying a question’s intent costs a few dozen tokens and lets factual requests go to a far cheaper tier — in my case roughly fifty to one. It sits behind a switch, it is measured per customer, and it can be turned off without a redeploy if quality moves.
- 03
How much of your information lives in tables?
It’s my first question on an unfamiliar corpus, and the answer doubles or halves the estimate. Flowing text settles quickly; 228,771 price-schedule rows needed an entirely separate processing chain, because in a grid the structure is the information.
- 04
Silence is a product goal
A system that always answers is a system that sometimes invents — the only truly serious failure mode on a contractual document. Making “that information is not in this file” a correct, deliberate answer is as much interface work as model work.
- 05
Scope by deliverable, not by days
“Delivered on the 30th” is a promise I can keep. “Two days a week” is a promise the unexpected breaks. Scoping happens by deliverable and date: that is what gets you a firm date instead of a theoretical availability.
Sectors
- Legal tech
- Contracts, clause extraction, case files
- Insurance
- Claims, policy and coverage comparison
- Healthcare and admin
- Records, coding, compliance
- Technical documentation
- Manuals, standards, specifications
- Finance
- Reports, regulatory filings, due diligence
The sector matters less than the situation: an input nobody wants to touch, stakes that rule out approximation, and a team too small to hire three specialists.
The demo works. Now what?
Describe the situation in three lines — what already exists, what has to exist, and by when. In thirty minutes I’ll tell you which of the three offers it falls into, or that it isn’t for me. Reply within 24 hours on weekdays.