NVIDIA Dory Looped Model: More Thinking, Same Facts

Diagram of NVIDIA Dory looping its 20-layer core several times

NVIDIA Dory is a four-billion-parameter model that turns ‘thinking longer’ into a quantity you can count. NVIDIA shipped it this autumn with a dial for how many times the model reruns its middle layers. That dial shows why raising an AI’s reasoning level sometimes helps and sometimes only inflates the invoice.

When an answer is wrong, teams tend to push reasoning from medium to high or buy a larger thinking budget. Occasionally it works. Frequently you spend more, wait longer and still get the wrong answer.

Thinking you can measure

Dory is built from 8 input layers, a 20-layer core and 8 output layers. Each token goes through the input stage one time, the core as often as you set, and the output stage one time. Models built this way are called looped transformers.

  • One loop: 36 layers stored, 36 run
  • Two loops: 36 stored, 56 run
  • Four loops: 36 stored, 96 run

With two or more loops, the designers say Dory beats some larger models on average over twelve benchmarks.

Stored size never shifts, and the weights on loop four are identical to those on loop one. Nothing is learned during the loops. The model polishes a temporary working state and discards it once the answer is finished. Extra loops mean extra computation for each token, yet they add no new knowledge.

Infographic: Dory stores 8 + 20 + 8 = 36 layers and executes 96 when its 20-layer core runs four times. Same weights, four passes.
Image: hackernoon.com

The Alice test

Take two statements: Alice beat Bob in a race, and Bob beat Cara. Ask who won, and the answer is present but must be pieced together, so more computation can repair a wrong response. Ask for Alice’s time, and the input simply doesn’t contain it. Four loops or forty leave the gap unchanged, while one database query would answer in milliseconds.

Michaล‚ Piszczek, a CTO who measures AI costs, proposes a quick check before you raise a reasoning setting: does the model already have the answer? Many real failures don’t. A model quotes an old contract version, lacks a current price or guesses what logs contain. More thinking can’t fix those, so paying for it is the costly error.

Infographic: Alice, Bob and Cara. Who finished first? More passes solve it. What was her time? No pass can create a missing fact.
Image: hackernoon.com

The price tag

Even a good fit has a cost. Fastino repeated two bands of middle layers in a 27B decision model and reported the trade:

  • It scored 0.56 points higher on a held-out test, with a 95% confidence range of 0.27 to 0.89.
  • Median response time grew from 83 ms to 112 ms, about 35% slower.
  • A third pass hurt results on a causal-reasoning benchmark.
  • The team tried about 110 setups, hit several dead ends and saw no clear benefit under one billion parameters.

Those numbers come from the vendor. They show that extra reasoning isn’t a staircase that only climbs: an additional pass may help, do nothing or damage an answer that was fine. Only the cost reliably rises.

Infographic: a loop has a price. +0.56 points on a held-out test; median latency 83 ms with one pass, 112 ms looped. Fastino GLiDE, vendor-reported.
Image: hackernoon.com

Four kinds of loop

People use the word loop for four separate things, and each is billed differently.

  • Looped transformer: repeats layers within one pass, leaves no trace, costs compute and delay per token.
  • Chain of thought: written reasoning, leaves a visible transcript, costs output tokens.
  • Agent loop: wraps tools around the model, leaves logs and side effects, costs calls, tool time and retries.
  • Recursive self-improvement: spans system versions, leaves a better system, costs research compute.

Only the agent loop imports new facts, as tools alone can reach beyond the model. The first two are pure thinking and can’t supply what’s missing.

Questions before buying more reasoning

Piszczek suggests four, in order. Does the problem pass the Alice test? If the answer isn’t in the input, retrieve it. Is the task about exactness? Sums and exact strings belong in code. Where does improvement stop? If two loops help and four don’t, use two. And what does one verified correct result cost, retries included?

Looped transformers do represent progress, since depth becomes adjustable instead of requiring a bigger file. Independent tests of Dory’s accuracy against its added delay are the next thing to look for.

Source: hackernoon.com

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.