Entry engagement
AI reliability
find out what breaks before your customers do.
You shipped an AI feature. It works in the demo, it works most of the time, and nobody on the team can tell you what it does on the bad 4%. We read it and write down what will break, why, and what fixing it costs.
Then you decide what to do about it — with us, with your own team, or not at all.
What gets reviewed
Six areas, every time. The scope does not expand mid-engagement and the price does not move.
Failure inventory
Every place the system can be confidently wrong. Not crashes — crashes are easy. The cases where it returns a plausible answer that is false and nothing in the pipeline notices.
The evaluation gap
What you measure today against what would actually catch a regression. Most teams have latency and error rate, and nothing that would tell them accuracy dropped after a prompt change.
Provenance trace
We take real outputs and try to answer “why did it say that” from your logs alone. Whatever we cannot reconstruct is what your first enterprise customer will ask about.
Irreversibility map
Everything the system can do that cannot be undone — send, charge, delete, publish, file — and whether a human sees it first. This is the section that tends to alarm people.
Cost model at volume
What the pipeline costs at ten times current throughput, which line item dominates, and where the ceiling should sit before the bill teaches you.
Unattended recovery
What happens when it breaks at 3am — rate limits, partial writes, poisoned queues, a provider outage mid-batch. Whether it resumes, stalls silently, or corrupts.
What a finding looks like
Each finding states the defect, the conditions that trigger it, what it costs you, and the smallest change that removes it. Written to be forwarded to an engineer, not read out in a boardroom.
Extraction confidence is computed but never gates anything
- Condition
- The extractor returns a per-field confidence score. The writer persists every field regardless of that score. On documents outside the training distribution — handwritten totals, rotated scans, a second currency — fields land in the database at 0.31 confidence and are indistinguishable downstream from fields at 0.98.
- Cost
- Silent. There is no queue of low-confidence rows for anyone to review, so the error surfaces when a customer disputes a number, which is the most expensive moment for it to surface.
- Smallest fix
- Persist the score alongside the value, add a threshold, and route anything below it to a review queue rather than to the table of record. Roughly two days. The threshold itself should come from a labelled sample, not from a guess.
The boundaries
What this is not
- Not a rebuild. We do not write production code during the audit.
- Not a retainer. It ends when the report is delivered.
- Not a sales document. If the system is sound, the report says so and it is short.
- Not a generic checklist. We read your code and your logs, not a questionnaire.
Who should not buy this
- You have not shipped the feature yet. There is nothing to read — talk to us about building it instead.
- You already have an eval harness, a provenance trail and a review queue. You do not need us; you need headcount.
- You want a second opinion to win an internal argument. We will give you an honest one, which may not be the one you wanted.
Most audits lead to a larger engagement. That is the honest commercial reason this is priced the way it is. It is not a condition of buying one.