What if LLM agents passed KV-cache to each other instead of text? I tried it -- 73-78% token savings across Qwen, Llama, and DeepSeek

If you've used multi-agent setups with LangChain, CrewAI, AutoGen, or Swarm, you've probably noticed: every agent re-tokenizes and re-processes the full conversation from scratch. Agent 3 in a 4-agent chain is re-reading everything agents 1 and 2 already chewed through. When I measured this across Qwen2.5, Llama 3.2, and DeepSeek-R1-Distill, 47-53% of all tokens in text mode turned out to be redundant re-processing.

AVP (Agent Vector Protocol) is my attempt to fix this. Instead of passing text between agents, it passes the KV-cache directly. Agent A finishes reasoning serializes its key-value attention states, and Agent B injects them. No re-tokenization, no redundant forward passes.

Text:    Planner -> [text] -> Critic re-tokenizes everything -> [text] -> Refiner re-tokenizes everything
Latent:  Planner -> [KV-cache] -> Critic injects, skips to generation -> [KV-cache] -> Refiner same

What it actually does:

• Same model on both sides? Direct KV-cache transfer, zero overhead.
• Same family, different size (e.g. Qwen2.5-7B talking to 1.5B)? Vocabulary-mediated projection. No learned params, no calibration data needed.
• Different families? Falls back to JSON. Not everything needs to be fancy.
• Transport-agnostic -- works alongside A2A, MCP, gRPC, whatever you're already using
• Binary wire format, not JSON+Base64 (33% overhead on tensor data is painful)

Numbers (these are structural, not accuracy claims):

Token savings of 73-78% and 2-4x speedups held consistent across all three model families. This isn't model-dependent -- it's just fewer forward passes, so less wall time. Here's the intuition: text prompt sizes balloon at each hop (186 -> 545 -> 1,073 -> 1,397 tokens in a 4-agent GSM8K chain). Latent stays flat at ~164-207 tokens per hop because prior context arrives as pre-computed KV-cache, not as text that needs re-encoding.

The gap widens with chain length. At 4 agents it's roughly 2x. At 16 agents (projected) it'd be around 6x, because text scales O(n^2) while latent scales O(n).

Limitations (yes, I know about these):

• Sample sizes are n=20 per model. The token and speed numbers are solid because they're structural (fewer forward passes is fewer forward passes), but n=20 isn't enough to make accuracy claims. That's future work.
• Tested on small models only (1.5B-3B on an RTX 3070 Ti). 7B+ results pending.
• This is a datacenter / same-machine thing. KV-cache for a 3B model runs about 130 MB per sample. You need 1 Gbps+ bandwidth minimum. Sending this over the internet is not happening.
• Requires KV-cache access, so self-hosted only. Won't work with OpenAI/Anthropic/etc. APIs.
• Same model only for now. Cross-model (Rosetta Stone) is implemented but not benchmarked yet.
• Latent uses 17-54x more VRAM than text because you're holding KV-cache across hops instead of discarding it. Totally fine for 1.5B-3B on 8GB+ GPUs. At 7B+ it becomes a real constraint, and I don't have a clean answer for that yet.

Try it yourself:

pip install avp

Two API levels depending on how much control you want:

import avp

msg = avp.pack("Hello", model="Qwen/Qwen2.5-7B-Instruct", think_steps=20)
answer = avp.unpack(msg, model="Qwen/Qwen2.5-7B-Instruct")


from avp import HuggingFaceConnector

connector = HuggingFaceConnector.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
context = connector.think("Analyze this problem", steps=20)
answer = connector.generate("Solve it.", context=context)

vLLM connector also available (pip install "avp[vllm]").

Links:

• SDK: github.com/VectorArc/avp-python (MIT, 377 tests, 7 benchmarks)
• Spec: github.com/VectorArc/avp-spec
• Benchmark details: BENCHMARKS.md

This is a nights-and-weekends project born out of my own multi-agent work. Happy to answer questions about the implementation and genuinely interested in feedback from people running multi-agent setups in production.