Every local-model project I start ends up needing the same thing before it does anything useful: a harness. Not a bigger model, not a longer prompt — a layer that decides what the model is even allowed to see at each step, and translates its output back into something a tool can act on without hallucinating a function that doesn’t exist.
Cloud models get away with dumping huge amounts of context at the problem and letting the model sort it out. Local models — Hermes Odysseus through LM Studio, mostly, for me — don’t have that luxury. Context is expensive, attention degrades over long windows, and a 7B or 13B model will confidently invent a tool call that looks exactly like the real one if you give it too much room to improvise. The harness is where you pay down that cost up front instead of debugging it later.
What the harness actually does
Three things, in practice:
- Scopes the context. Godot, Unreal, and Blender all expose enormous APIs. Handing the whole surface to a local model is how you get plausible-sounding nonsense. The harness narrows what’s visible to whatever the current task actually touches — the current scene’s nodes, not the whole engine.
- Abstracts tool calls down to results. The model doesn’t need to see every intermediate step of a Blender operator running. It needs “here’s what changed” back, in a shape it already knows how to reason about. Verbose tool narration is where a local model’s limited context gets eaten alive.
- Holds state across turns. This is the part that took the longest to get right. A harness that resets context every call is just a fancy autocomplete with extra steps. The useful version remembers what it already tried, what failed, and why — so the model can behave like it’s making progress on a problem instead of re-discovering it every message.
Where this is going
Right now this lives as three separate implementations — one per engine — because the tool surfaces are different enough that a shared abstraction kept fighting me. I’m not sure that’s wrong yet. It might just be that “context engineering for local models” is genuinely engine-shaped, not tool-shaped, and the generalizable part is smaller than I originally assumed: scope tightly, summarize aggressively, never let the model narrate.
If that changes — if a shared harness pattern actually holds across Godot/Unreal/Blender — that’s a portfolio piece. Until then, this is where it lives.