looped aikitchen - a looped transformer lab

looped.aikitchen.cc github

NVIDIA released its looped model Dory1.0-4B with no tech report. This lab rebuilds the research the market already knows from smaller looped models, runs the new kid locally in Docker, and measures loop counts on real GPU.

Status: the stack is running. A looped llama.cpp build of PR 30306, a 3.42 GiB GGUF on an RTX 4070, loop-count variants from GGUF metadata surgery, a smoke test, and a replicated 36 task synthetic loop eval.

Looped transformer, kitchen art

A looped transformer is a new paradigm for how a model thinks. A classic transformer pours every token through its stack once, deep and expensive. A looped transformer keeps a small stack and walks it in a circle: the same block runs two, three, four times, each pass refining what the last pass left in the latent state - thinking longer without paying for more weights. The loop count becomes a runtime dial: turn it up and the model gets smarter answers at fewer tokens per second, turn it down and you get speed. One checkpoint, many effective depths, and you pay for depth on your own GPU, at inference time, per question.

Second looks are a hot topic right now, and this lab is only half of it. The same trick, run against retrieval instead of weights, is more explored on our sibling lab refeed.raptor.today - the cost of looking twice, with visible traces and published metrics.

What the lab learned so far

The one sentence version: going from 1 to 2 loops buys the most accuracy (+15 points on a 72 question combined eval) while deeper loops mostly just cost speed. Stack programs are the loop sensitive kind (5/24 at one loop, 11/24 at two); easy arithmetic almost never needs a loop.

Task type1 loop2 loops3 loops4 loops
arithmetic420/2421/2422/2423/24
graph_reach18/2422/2419/2420/24
stack_program5/2411/2411/2411/24
all43/7254/7252/7254/72

Two independent data draws at temperature 0, 12 GiB GPU. The same accuracy buys speed: throughput falls from about 110 tok/s at one loop to 42-47 at four.

How the loop knob was built

The dory support in llama.cpp (PR 30306, still open) reads the loop count from GGUF metadata only (dory.num_loops, uint32), no CLI flag. The lab parses the exact on disk layout, finds the field by scanning all 44 metadata KVs (this GGUF is v3 with no alignment padding), copies the file and rewrites a single uint32, so weights are byte identical and variants are cheap: 1, 2 and 3 loop clones of the official 4 loop GGUF.

Serving itself is its own story: the official llama.cpp v0.8.0 image says unknown model architecture: 'dory', so the lab builds its own CUDA image pinned to the PR head.

Deep dives

One file per topic, all hand written, with the numbers from this lab. On GitHub they live in mds/.

DocWhat is inside
mds/looped-research.md the looped transformer lineage mapped: Universal Transformers, LoopedLAN, Huginn, Ouro, Hyperloop, Loopie, the KV cache fix family, and what each paper actually measured
mds/datasets.md public code, harnesses and dataset inventory from every looped paper, plus our own synthetic loop eval design and the live results
mds/lab-report.md the full hands on story: from unknown model architecture: 'dory' to a custom CUDA build, loop count surgery on GGUF metadata, and a replicated 4 point loop curve with honest caveats

Run it

The lab expects a pixi workspace named loop, a CUDA host with about 8 GiB free VRAM, and a read-only HF token in .env.

  1. python tests/make_dory_loop_variants.py --cycles 1 2 3 - the loop count clones
  2. docker build -t loop-llama-cpp:pr30306 docker/loop-llama-server and docker compose -f docker/loop-llama-server/compose.yaml up -d - serve on port 8777
  3. python tests/test_dory_smoke.py - 3 checks against the live server
  4. python tests/test_loop_evals.py --cycles 1 2 3 4 - borrow the GPU per variant, score, hand it back

Notes

Everything here ran on one RTX 4070 and is reproducible from a clean clone. The model license is NVIDIA Open Model, the local code is for learning, and the results are the lab's own, not NVIDIA's.

  • No cloud model and no API key. The only network calls at runtime are the local llama.cpp server and the read-only HF Hub for downloads.
  • Static page only. Everything dynamic runs in the loop- prefixed Docker stack; the GUI, when it exists, builds into gui/, never here.