webSearch / webExtract — ports over fetch

Two tools — search the web, extract a page — built as a thin dispatch kernel over an injected WebBackend port, with an optional Compressor seam for token-saving LLM summarization. The kernel resolves nothing itself: it validates the request, calls the injected backend, re-validates the untrusted result with Zod, and returns a Result. The production backend is a dependency-free fetch impl. A CLONE of Hermes' web_tools.py (1378 LOC).

Two ports, one kernel

The source builds around two seams — a provider registry and an optional LLM-compression pass. Both become injected ports; this module imports no SDK and no concrete backend:

// packages/hermes/src/web/types.ts:120-140
export interface WebBackend {
  search(query: WebSearchQuery): Promise<Result<readonly WebSearchResult[], Error>>;
  extract(url: string): Promise<Result<WebExtractResult, Error>>;
}
export type Compressor = (
  text: string, instruction: string,
) => Promise<Result<string, Error>>;        // optional; absent ⇒ raw content

A network failure is err; an empty search is ok([]) — the difference is preserved in the type, exactly like the source's empty data.web. The Compressor mirrors the ReviewProposer seam from the learning loop: "a model call in prod, a fake in tests."

Defense in depth: map defensively, then re-validate

The flow has two boundaries. The fetch backend maps untrusted JSON defensively (unknown shapes → empty strings); then the kernel re-validates every row with Zod, failing the whole call closed on a single bad row:

// packages/hermes/src/web/web.ts:69-90 — webSearch
const parsedQuery = webSearchQuerySchema.safeParse(query);
if (!parsedQuery.success) return err(new Error(`Invalid web search query: …`));
const found = await deps.backend.search(clampQuery(parsedQuery.data)); // clamp [1,100]
if (!found.ok) return found;
const validated: WebSearchResult[] = [];
for (const raw of found.value) {
  const parsed = webSearchResultSchema.safeParse(raw);   // untrusted backend row
  if (!parsed.success) return err(new Error(`Invalid web search result: …`)); // 1 bad ⇒ fail all
  validated.push(parsed.data);
}
return ok(validated);

The clamp, and the empty-vs-error distinction

maxResults is clamped to [1, 100] before the backend ever sees it — the source's min(max(limit, 1), 100):

// packages/hermes/src/web/web.ts:134-139 — clampQuery
const clampQuery = (query) => {
  if (query.maxResults === undefined) return query;   // absent ⇒ pass through
  const clamped = Math.min(Math.max(query.maxResults, 1), 100);
  return { ...query, maxResults: clamped };
};

The test "clamps maxResults into [1, 100]" sends 9999 and 1 and asserts the backend sees exactly [100, 1]. An empty result set is ok([]), not an error — "no hits" is a successful search, distinct from "the provider is down" (err).

Optional compression — gated by a size floor

For extracts, an injected Compressor can shrink large pages to save tokens. It's skipped below a size floor (compressing small content wastes a model call) and absent by default:

// packages/hermes/src/web/web.ts:118-132 — maybeCompress
const minLength = deps.compressMinLength ?? DEFAULT_COMPRESS_MIN_LENGTH;  // 5000
if (compressor === undefined || result.content.length < minLength) {
  return ok(result);                            // no compressor / too small ⇒ raw
}
const compressed = await compressor(result.content, instruction);
if (!compressed.ok) return compressed;          // compressor failure ⇒ err (fail closed)
return ok({ ...result, content: compressed.value });

Common confusions