pages.stencel.ai
software as a thought
KV-cache sharing between models

Sharing the cache,
not the sentence.

Two AI agents collaborate today by writing each other English. One decodes its understanding into prose, token by token; the other re-reads the prose and rebuilds the understanding from scratch. A new line of research asks whether models can hand over the internal state directly — the cache, not the sentence — and what breaks when they try.

tokens flow in → "The capital of France is …" one column per token L31 L30 ⋮ every layer keeps its own K and V ⋮ L2 L1 L0 K V
Fig. 01 — The cache as sediment. As a model reads context, every transformer layer deposits a key vector and a value vector per token. The accumulated strata are the KV cache: a model-specific, position-stamped record of how this model read this text.

The problem, stated plainly. In a multi-agent system, Model A finishes analysing something and must tell Model B. Today that means: A autoregressively decodes hundreds of tokens of English → the tokens cross a wire → B tokenizes them → B runs a full prefill → B reconstructs, approximately, an understanding A already held in its cache. We serialise a representation into prose, then reverse-engineer the representation from the prose — a decode tax, a prefill tax, and a lossy compression through natural language, on every handoff.

The obvious question: A's understanding already exists as tensors. Why not ship the tensors? The answer turns out to split into one solved problem, one genuinely hard problem, and one idea that might change how agents are built.

01

What the cache actually is

Inside every transformer layer, the hidden state h_i of token i is projected three ways:

q_i = h_i·W_Q   // what's relevant to me right now?
k_i = h_i·W_K   // my address in semantic space
v_i = h_i·W_V   // the content stored at that address

Attention is a soft database lookup — the new token's query is matched against every stored key, and the result is a weighted blend of the stored values:

Attention(q_t, K, V) = softmax(q_t·Kᵀ / √d) · V
q_t · new token the search request k₀ "The" k₁ "capital" k₂ "of" k₃ "France" KEYS — the index v₀ v₁ v₂ v₃ VALUES — the records match weigh blend → output
Fig. 02 — Attention as lookup. Query = search request. Keys = index. Values = records. Queries are needed once and discarded; keys and values must survive, because every future token will search against them. That is what gets cached.

Stored per layer as tensors shaped [batch, kv_heads, seq_len, head_dim], the cache turns generation from "recompute the whole history for every new token" into "compute one new column, append, attend." Three properties matter for everything that follows:

It is not the text. It's a processed representation of the text. It is not quite the model's "thoughts" — those are the hidden states h; the cache is the attention-facing projection of them, the part packaged for future lookup. And it is written in the model's private coordinate system — a fact that will shortly become the whole problem.

02

The easy case: identical twins

If two servers run the same model — same weights, same everything — the cache is portable state. Server 1 prefills a 50,000-token document once; the KV blocks move over the wire; Server 2 continues decoding as if it had read the document itself.

SERVER 1 · Model A prefilled once · 50k tokens CPU · RDMA · NVMe KV blocks in flight SERVER 2 · Model A continues decoding · zero re-prefill
Fig. 03 — Distributed prefix caching. This is production reality, not research: it's the mechanism behind prefix caching, disaggregated prefill/decode, and the LMCache / Mooncake / NVIDIA Dynamo class of systems.

The reason it works is that everything the cache implicitly assumes still holds on the receiving side:

same weights same architecture & layer count same tokenizer & token sequence same KV-head config same positional encoding & positions same dtype (or a supported cast) same engine cache layout

Break any one of those, and you've left the easy case.

03

The wall: different models

Now let Model B have different weights. Both models read the word "bank", both produce a key vector of the right shape — and the vectors mean completely different things, because W_K differs, so each model has laid out its semantic space along private axes.

MODEL A reads "bank" k = [ 0.8, −0.2, 0.1, … ] its own coordinate frame MODEL B reads "bank" k = [ −0.1, 0.5, 0.7, … ] a different coordinate frame same shape · different meaning like handing an ARM binary to an x86 core: right bytes, wrong instruction set
Fig. 04 — Private coordinate systems. A KV cache is machine code for one specific machine. Verbatim reuse across models corrupts meaning even when the tensors physically fit.
Everything that can disagree
axisModel AModel B
layers3248
kv heads328 (GQA)
head dim128256
positional encodingRoPE, θ config ARoPE, θ config B
tokenizer["Kubernetes"]["Kuber","netes"]
coordinate frameprivateprivate
The partial exception — DroidSpeak (2024). If B is a fine-tune of the same base as A — same architecture, weights only nudged — much of the cache survives. DroidSpeak profiles, per model pair, which layer groups are sensitive, recomputes only those, and reuses the rest: up to 2.78× faster prefill with negligible quality loss. Sibling models can share memories. Strangers can't. Yet.
04

The inconvenient arithmetic

Before any translation cleverness, there's a logistics problem: caches are enormous. For a standard decoder,

cache bytes ≈ 2 × layers × tokens × kv_heads × head_dim × bytes_per_elem  // 2 = K and V
precision
4.2 GB
Fig. 05 — The size gap, log scale. The state you'd like to transfer is ~six orders of magnitude bigger than the sentence it replaces. Over commodity links, shipping the raw cache can be slower than decoding and re-prefilling the text — which is why every practical design leans on compression, quantisation, selective layer/token transfer, or RDMA-class fabrics.
05

The research ladder, 2024 → 2026

Four rungs, each relaxing one constraint of the previous:

Nov 2024
NSDI '26

DroidSpeak arXiv 2411.02820

Constraint: same base architecture. Fine-tunes of one foundation model reuse most cache layers; per-pair profiling picks the sensitive layer groups to recompute. Up to 2.78× prefill speedup, negligible accuracy loss.

Oct 2025
ICLR '26

Cache-to-Cache (C2C) arXiv 2510.03215

Constraint: same context, but truly different models. A learned neural projector fuses the source cache into the target's, with per-layer gates choosing where the transfer helps. Reported ~3–5% higher accuracy than text exchange at ~2× lower latency.

Jan 2026

Latent Space Communication via K-V Cache Alignment arXiv 2601.06123

Idea: one shared interlingua. Frozen base models; small adapters translate each model's cache into and out of a common aligned space — a high-bandwidth channel, and even learned skills like soft prompts become transferable across models.

May 2026

Latent Cache Flow (LCF) arXiv 2605.22863

Constraint relaxed: different contexts. Keys and values are jointly translated and compressed through a low-dimensional latent bottleneck — adapters at ~4% of C2C's size — and the LCF-X variant transmits a summary of only the information the receiver doesn't already have.

Also on the map: KVCOMM · NeurIPS'25 — anchor-based offset correction for differing prefixes/LRAgent · ICML'26 — multi-LoRA agents share the base cache, keep low-rank deltas/LatentMAS — training-free latent collaboration via shared cache working memory/LCGuard — adversarial hardening so shared caches can't be decoded back into prompts

These are research results, not production guarantees — reported on specific model pairs and benchmarks. The trendline, though, is unambiguous: every six months, one more constraint falls.
06

How translation actually works

The shared shape of the C2C / aligned-space / LCF family: encode the source cache into a communication latent, decode it into the target's coordinate frame, then fuse it with whatever the target already knows — with learned gates deciding, per layer, how much to trust the import.

K_A , V_A source cache E_A select · compress strip source positions Z latent packet small · position-free D_B align layers · restore pos gate g_l per layer B's own cache B attends
Fig. 06 — The translator. Base models stay frozen; only the adapters (E, D) and gates are trained — against reconstruction, attention-alignment, logit-alignment, and end-task losses. Fusion itself can be concatenation, gated addition, cross-attention, or injection into selected layers only.

Same context vs. different context

If both agents processed the same tokens, translation can map cache entries position-for-position — tractable, and where C2C lives. Real agents, though, hold different private contexts: different lengths, different positions, overlapping and disjoint knowledge. Token-aligned translation stops making sense. The newer pattern (LCF-X) is to extract only what's new to the receiver, compress it into semantic packets, and inject it as virtual memory — closer to a briefing than a memory transplant.

07

Where it bites

Six problems stand between the papers and your platform:

Positions

RoPE bakes position into every cached key. Move an entry from position 8,000 to 2,000 and its meaning corrupts — positions must be remapped or stripped in transit.

Layer alignment

Layer 12 of A is not layer 12 of B. Depth of processing differs, so translators learn many-to-many mappings — A's layers 8–14 might feed B's 16–24.

Tokenizers

A sees ["Kubernetes"], B sees ["Kuber","netes"]. Token-by-token correspondence is unreliable by construction.

Reuse ≠ recompute

A transplanted cache records how tokens were read under the source context. They never attended to the target's prompt — so reuse is an approximation of joint prefill, not an equivalence.

Security

A transferred cache is executable semantic state: cache poisoning, latent prompt injection, cross-tenant leakage, prompt reconstruction. A serious runtime needs provenance, tenant isolation, integrity checks, and adapter authentication — LCGuard-style adversarial hardening is the first work aimed squarely at this.

Observability

You can log text. You can't audit four billion floats. Regulated environments will likely demand a text shadow of every latent exchange — efficiency on the wire, English in the evidence log.

08

Two problems wearing one name

The most useful thing I can leave you with: "KV-cache sharing" is two different ambitions that happen to use the same tensors.

Goal 1

Avoid recomputation

"B has the same context as A — stop paying the prefill twice." An inference-systems optimisation: cheaper, faster, quality-neutral by design.

prefix caching · disaggregated serving
DroidSpeak · KVCOMM · LRAgent
Goal 2

Transmit knowledge

"A understands something B doesn't — inject the understanding itself." Not an optimisation: a machine-native communication protocol, with its own semantics, failure modes, and threat model.

C2C · aligned latent spaces · LCF
the beginning of something new

Text transfers conclusions.
The cache is the representation that produced them.