Latent Bridge: Asking for Help Without Words

Anonymous authors

Frontier-model prefill. Edge-scale decoding.

Sender: Qwen3.8 Max · GLM 5.3 · DeepSeek V4 Pro
Receiver: Qwen3.5-4B — a compact, edge-capable model

Sender–receiver collaboration through a latent bridge The same prompt goes to both models. A frontier-scale sender, such as Qwen3.8 Max, GLM 5.3, or DeepSeek V4 Pro, performs prefill only. A trainable bridge maps its hidden states to latent attention keys and values for Qwen3.5-4B, a compact receiver capable of running on suitable edge hardware. The receiver generates all output tokens. In this illustrative animation, the prompt asks: If x + 1/x = 3, find x² + 1/x². The original prompt and the bridge's latent keys and values converge on the receiver together, representing joint conditioning rather than exact transfer timing. The receiver then reveals the answer: 9 − 2 = 7. Token boundaries are schematic. Prompt If x + 1/x = 3, find x² + 1/x². Frontier sender Latent bridge 4B receiver Prefill only Hidden states → KV Prompt + latent KV No text generated All output tokens Illustrative token boundaries
The sender helps through latent representations, not an intermediate text response.

Abstract

We introduce POND (Prefill-Only, No-Decode), an architecture that separates large-model prefill from small-model decoding. A frontier-scale sender—Qwen3.8 Max(2.4T), GLM 5.3(753B), or DeepSeek-V4-Flash(284B)—processes the input without generating text. A learned bridge transfers its hidden states as additional attention keys and values to Qwen3.5-4B, a compact receiver capable of running on suitable edge hardware. The receiver generates the complete answer using both the original prompt and the latent guidance. Optional periodic review lets the sender read the partial response and append updated guidance while preserving the receiver’s existing KV cache. Both base models remain frozen; only the bridge is trained.

Inference Examples

Math and code examples are forthcoming.