Kimetsu
Kimetsu Network

Results

What one Kimetsu brain adds, and what each agent keeps when memory is merged over the network.

Results

These are the measured gains from one Kimetsu brain, and from merging memory across a network. The harness is kimetsu-bench (network-bench/ and gpu-suite/).

One Kimetsu brain vs no memory

Model and benchmarkNo memoryWith KimetsuImprovement
Qwen 7B, LoCoMo22.840.3+77% (1.77x)
Qwen 0.5B, LongMemEval oracle5.721.4+273% (3.7x)
Qwen 0.5B, LongMemEval s-cleaned5.420.3+279% (3.8x)
Qwen 0.5B, LoCoMo (answerable questions)2.122.1about 10x

Network: each agent alone vs merged memory

Model and benchmarkAloneMergedImprovement
Qwen 7B, LoCoMo31.838.9+22%
Qwen 7B, LongMemEval oracle11.127.9+151%
Qwen 0.5B, LoCoMo (answerable questions)16.221.9+35%
Qwen 0.5B, LongMemEval oracle14.123.4+65%
Qwen 0.5B, LongMemEval s-cleaned14.821.3+44%

QUIC transport keeps about 99% of the gain (7B LoCoMo 40.1 vs 40.3 in-process; 0.5B LongMemEval oracle 22.7 vs 23.0 directory sync).

Caveats

  • LongMemEval scores are proxy token F1, not the official LLM judge.
  • QUIC runs were single-machine loopback.
  • LoCoMo rows marked "answerable questions" exclude the adversarial category 5, because in the full LoCoMo score the 0.5B no-memory baseline looks better only by abstaining on those.
  • Hybrid retrieval results are not included pending a rerun after issue #68.

Qwen 7B runs on an RTX 5060 Ti 16 GB. Qwen 0.5B runs on CPU and GPU.

On this page