Hunter-Seeker Vs Fable 5 Vs Fable 5 + Tuned GBM
Customer 9497-QCMMS is about to cancel. We asked three AI systems to study the same telecom company’s spreadsheet — 7,043 customers, their contracts, their tenure, their bills — and all three put this customer at the top of the risk list. That is where the agreement ends.
Ask each one why, and you get three different kinds of answer: a probability with no interior, a hunch dressed as arithmetic, and a case file.
each method ranks the rest
measure · concentration in top 10%
lift 3.0 · shortlist 3× richer than random
verifier · fresh-context, artifacts only
checks · 19 protocol + condition isolation
A fresh-context verifier re-derived every number on this page from artifacts alone.
It answers from its own judgment, and nothing is validated.
rubric: month-to-month contract +0.30 tenure < 6mo +0.25 electronic check +0.15 # weights chosen by intuition # no holdout, no calibration, no guarantee
It returns a number with no reasoning attached.
{ "customerID": "2636-ALXXZ",
"churn_probability": 0.9761 }
# a number from a 100-tree ensemble
# per-customer explanation: noneIt returns a case file.
Illustrative shape. On a cleared finding top_decile_lift, top_factors and leak_guard populate; on a non-finding they are null — the honest-null contract.
"entities": [{ "entity_id": "9497-QCMMS", "score": 0.9219, "tier": "critical" }], "top_factors": ["higher usage_score", "lower support_tickets"], "top_decile_lift": 2.96, "gate_verdicts": ["holdout lift 2.96 ≥ 1.5 surface bar"], "calibration": { "coefficient": 1.01 }, "leak_guard": [], "provenance": { "engine_version": "0.2.2", "core_hash": "4d24e9fc…" }
Accuracy is where the comparison starts.
Hunter-Seeker matched or edged every challenger on lift — statistically even with the black-box model an agent trains for itself, and 1.8× ahead of Fable 5’s unaided judgment — while remaining something a human can read.
| Method | Loan (100K) lift | Telco (real) lift | Answers “why?” |
|---|---|---|---|
| Fable 5 — judgment only | 2.07 | — | partially — unvalidated |
| Fable 5 + Trained GBM | 4.77 | 2.90 | no |
| Hunter-Seeker (scorecard) | 4.78 | 2.96 | yes — every point |
Five questions, three report cards.
The last two rows cover reproducibility and the audit record.
| The question an operator asks | Fable 5 | Fable 5 + Trained GBM | Hunter-Seeker |
|---|---|---|---|
| Who should I focus on? | yes | yes | yes |
| How confident are you? | no | yes | yes |
| Why this one? | partially | no | yes |
| What would change it? | no | no | yes |
| Can you prove it to an auditor? | no | no | yes |