Lesson 3 · Adversarial design
The gold set: 40 adversarial cases before any scale
Before touching the full corpus, the pipeline had to survive 40 hand-audited cases designed to break it — certified 40/40 by two independent validators before extraction was even written.
Real signals
10 bookmarks + 10 transcripts with proven pain, jobs, demand, WTP and workarounds — every receipt re-opened line by line.
Same-root pairs
The same underlying entity in both families (bookmark + transcript) — two provenance nodes, ONE independent source.
Promotional
Creator funnels and self-declaration. Market-billion promises stay SOURCE_CLAIM; they never become customer demand.
Irrelevant
Memes, gossip, game-design lectures. The honest answer is no_signal — abstention is a first-class state.
Contradictory
"SaaS is dead" vs "SaaS is minting millionaires". Both sides must surface with explicit conflicts[] — never averaged away.
Certified
62/62 receipts verbatim; every same-root pair proven to be the same entity, not just the same theme.
Negative controls that must always hold
| Control | Expected |
|---|---|
| Irrelevant package | zero signals |
| Promotional billion-dollar claim | SOURCE_CLAIM only, never demand |
| 100 items from the same creator | no score elevation |
| Missing receipt or ICP | fails closed |
| Shuffled input order | identical ids, ranking and bytes |
| Corrupted receipt | named failure, never silent |