Lesson 4 · Extraction

The extractor: receipts, dedupe and honest abstention

8.820 signal rows from 2.537 packages — every one with a verbatim quote bound to an exact byte span. 1.225 are alias rows: duplicates tracked forever, counted once.

What a signal looks like

{
  "signal_id": "sig_…",
  "kind": "pain | job | workaround | demand | willingness_to_pay | …",
  "evidence_quote": "Ele quer o problema dele resolvido e ponto.",
  "locator": { "file": "Transcripts/YouTube/items/…/source.md",
               "line_start": 68, "byte_start": …, "byte_end": … },
  "speaker_role": "narrator | author | quoted_third_party",
  "truth_class": "SOURCE_CLAIM",
  "dedupe_key": "…", "conflicts": []
}

Scaffold removed

Every package carries a generated wiki contract ("make contradictions or gaps explicit"). Extraction keeps only the title block and the Full Cleaned Content|Transcript body — boilerplate never becomes a market signal.

One evidence, one slot

The same sentence in source.md AND raw/transcript.txt is ONE piece of evidence. Across packages, the first occurrence is canonical; the rest become alias: rows with an alias_of pointer.

Abstention is honest

476 packages are no_signal with a named reason (pure promotion, noise context, no candidates). 6 are blocked (source missing) — named, never silent.

Guards learned the hard way

Guard	Why it exists
Word boundaries	"whatever" is not hate, "Mythos" is not a myth, "captions" is not a cap, "marketer" is not a market, "Conflict" is not the NFL
False-friend anchors	PT "isso demanda tempo" is not market demand; "já era melhor" is not a death claim
Negation stripping	"sem problema" and "no human bottlenecks" are not pain
Author role	"they paid me $10,000" is a creator self-claim — never customer WTP
Question guard	"how do I use Claude?" is a question, not adoption
Idiom guard	"dead simple" is not a death; "work full time" is not product failure