Turn the local cluster into a fast, private preprocessing factory that prepares the work for Frontier models.
report-cluster-ai.html in the workspace — the interactive document that consolidated the research and originated this course.
Frontier models are powerful but expensive and high-latency. The local cluster is a preprocessing factory: it ingests raw text, images, audio and documents, filters noise, enriches context, and hands a small, decision-ready payload to the Frontier model. Most of the work never leaves the Mac.
Think of a research assistant: before a senior expert writes an opinion, an assistant gathers sources, highlights contradictions and drafts a one-page brief. The expert still decides, but the hard preparation is done locally.
The factory runs as a set of local HTTP endpoints: mlx_lm.server for text/code, mlx_vlm.server for vision, and mlx_audio pipelines for speech. A lightweight orchestrator — Python, LangGraph or n8n — routes chunks through RAG (pgvector + bge-m3), local inference, and validation gates. Only the fused, validated brief is sent to the Frontier API.
Loop engineering is a discipline: define a measurable goal, run one small unit at a time, verify at the real boundary, and iterate. It is model-agnostic and works with any local or cloud model.
AI Developer Workflows are tools such as Codex, Claude Code or Kimi Code that edit files from natural language. They can run inside a loop, but they are not the loop itself.
The local cluster is the ideal place to host the loop: fast feedback, private data, and low cost per iteration.
Loop engineering on the cluster can look like this: a sidekick.py script calls a local model to draft a plan, runs the plan through a real command, captures the output, and feeds it back to the model for correction. AI Developer Workflows accelerate the coding step, but the verification step always hits the real boundary — a running server, a unit test, or a JSON schema check.
Multimodal preprocessing can be mapped to seven phases. The local cluster handles phases 2–6, leaving the final synthesis to the Frontier model.
Each phase has a real boundary: ingestion reads a file, cleaning runs a parser, enrichment calls an embedding model, synthesis calls a local LLM, validation runs a gate, and handoff produces the payload for the Frontier API. The loop repeats until the handoff passes.
Do not write a long specification first. Build a throwaway script that reads one real file, calls one local endpoint, and prints one result. The shape of the data will teach you more than a blank document.
Once the prototype works, extract the reusable parts into a spec. The spec emerges from working code, not the other way around.
A prototype can be a single Python file that calls http://localhost:8080/v1/chat/completions with a small prompt, then calls http://localhost:8000/v1/chat/completions for vision. Run it, inspect the output, tune the prompt, and only then wrap it in a class or workflow.
import requests, json payload = { "model": "mlx-community/Qwen3.6-35B-A3B-8bit", "messages": [{"role": "user", "content": "Summarize this log in one sentence."}], "temperature": 0.3 } r = requests.post("http://localhost:8080/v1/chat/completions", json=payload) # Inspect the result, then iterate on the prompt and gates. print(r.json()["choices"][0]["message"]["content"])
Imagine a top-tier intern who reads every background document, flags risks, checks numbers, and drafts a one-page recommendation. The CEO reads only the brief and makes the decision.
The local cluster is that intern. It does not replace the CEO — the Frontier model — but it makes the CEO vastly more productive and the meeting much cheaper.
The "brief" is a structured JSON object: context, options, risks, recommendation and a confidence score. It is generated by a small local model such as Qwen3.6-35B-A3B, validated with JSON schema, and only then passed to the Frontier API. The Frontier call is short because most context has been distilled.
Test what you learned with quick exercises.