Every figure the AI reports, ticked back to the line that proves it.
Auditors trace each number in a working paper to its source and tick it. This tool does the same for AI that reads a practice's financial statements during an acquisition. A figure is accepted only when the line it cites really shows it. Anything else becomes an exception, even when the number happens to be right.
Working paper A-1
| Item | Reported | Tick | Source |
|---|---|---|---|
| Turn on JavaScript to work through the schedule. | |||
NORTHGATE CLINICAL GROUP Consolidated Balance Sheet As at 31 December 2024 Amounts in US dollars ASSETS Cash and cash equivalents 1,204,500 Trade and other receivables 3,410,000 Total current assets 4,614,500 Property, plant and equipment 8,890,000 Total assets 13,504,500 LIABILITIES Trade and other payables 1,120,400 Long-term borrowings 4,200,000 Total liabilities 5,320,400 EQUITY Share capital 1,000,000 Retained earnings 7,184,100 Total equity 8,184,100 Total liabilities and equity 13,504,500 CONSOLIDATED STATEMENT OF OPERATIONS Revenue 24,880,000 Cost of sales (14,920,000) Gross profit 9,960,000 Net income 2,415,000
- Agreed to source. Right, and the cited line shows it.
- No source given. Right, but a reviewer must find it again by hand.
- Exception. Wrong, or the cited line does not back it up.
- Not reported. Left blank.
Review summary
6 statements, 8 figures each: 48 figures per preparer. Only figures agreed to source are counted as usable, because they are the only ones a deal team can rely on without checking them again.
| Prepared by | Agreed to source | No source given | Exception | Not reported | Distribution | Usable |
|---|---|---|---|---|---|---|
| Current versionFinds each figure, applies the statement's units, and points to the exact line. | 48 | 0 | 0 | 0 | 48 of 48 | |
| Previous versionPoints to its sources, but confuses lines with similar names. | 20 | 0 | 22 | 6 | 20 of 48 | |
| First draftGrabs the number after a label and never says where it came from. | 5 | 11 | 8 | 24 | 5 of 48 | |
| Test: no sourcesBuilt to fail. Every figure correct, and not one source given. | 0 | 48 | 0 | 0 | 0 of 48 | |
| Test: fabricatorBuilt to fail. Invents believable figures and cites the row label, so each one looks sourced. | 0 | 0 | 36 | 12 | 0 of 48 |
The "no sources" test gets every figure right and is still usable on none of them. Scoring correctness alone would have rated it perfect.
It also catches a statement that does not add up
Every figure on Northgate Clinical Group's statement was read and agreed to source. The statement still does not foot: liabilities plus equity should equal total assets, and here they fall short. A tool that only copied numbers would carry the gap straight into the deal summary.
| Total liabilities | 5,320,400 |
|---|---|
| Total equity | 8,084,100 |
| Liabilities and equity | 13,404,500 |
| Total assets | 13,504,500 |
| Difference | Exception(100,000) |
Does not foot by 100,000. Flagged before it reaches the deal team.
No update ships if it makes a single figure worse
Every change to the reader, including a change to an AI prompt, is worked through the same statements before it ships. If any one figure gets worse, sign-off is refused, even when the overall score goes up.
Previous version to current version
- Figures fixed
- 28
- Figures made worse
- 0
Approved to ship
The same change, rolled backwards
- Figures fixed
- 0
- Figures made worse
- 28
Refused
The same discipline for the data room
A sample diligence database, seeded on purpose with one problem of each kind, so every check should find something. The test suite also proves each check finds nothing once its problem is removed, so a check cannot pass by always raising the alarm.
| Planted problem | Result |
|---|---|
| Figures with no document behind them | Found 1 record |
| Records that point at things that do not exist | Found 2 records |
| The same file uploaded twice for one practice | Found 1 record |
| Checklist items with nothing filed against them | Found 1 record |
| Totals from two sources that disagree | Found 1 record |
| A marketing asset carrying another brand's name | Found 1 record |
Supporting schedules for engineers
Grounding means the cited span is sliced out of the document and re-read: a citation that does not contain its own value fails, and the offending text is recorded. Grounded recall is the share of true values returned correct and grounded, and it is the headline metric.
| Extractor | Recall | Precision | Citing | Grounding | Grounded recall |
|---|---|---|---|---|---|
rules:v1 | 33.3% | 66.7% | 25.0% | 25.0% | 10.4% |
rules:v2 | 62.5% | 71.4% | 100.0% | 71.4% | 41.7% |
rules:v3 | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% |
adversarial:uncited | 100.0% | 100.0% | 0.0% | 0.0% | 0.0% |
adversarial:fabricating | 0.0% | 0.0% | 100.0% | 0.0% | 0.0% |
Release gate: rules:v2 to
rules:v3, PASS
| Metric | Baseline | Candidate | Delta |
|---|---|---|---|
micro_recall | 62.5% | 100.0% | +37.5% |
micro_precision | 71.4% | 100.0% | +28.6% |
micro_f1 | 66.7% | 100.0% | +33.3% |
grounding_rate | 71.4% | 100.0% | +28.6% |
grounded_recall | 41.7% | 100.0% | +58.3% |
citation_rate | 100.0% | 100.0% | 0.0% |
The gate fails on any regressed cell, any drop in a headline metric, any change to the golden set itself, and any arithmetic check that stops reconciling.
Live model on this instance: deepseek-v4-pro via https://api.deepseek.com.
make install && make ci runs the linter, the test suite, every
extractor, the regression gate and the SQL checks. make compare exits non-zero on
any cell that got worse. The documents are synthetic and every figure is fictional; the golden
set is generated by tools/make_golden_corpus.py so it cannot drift.