Karpo has open-sourced karpo-browser, a browser agent that splits web tasks between a general language model and a small, fast decision model. On the Browserbase Benchmark v2, the browser agent scored 89% accuracy across all 102 tasks at 201 seconds and $0.096 per task.
The company also released the weights of its decision model, Karpo Decision 27B Act, and a new benchmark called KBB. The design comes from the work behind Karpo’s Personal Agent, which books tickets, places orders and looks things up for users.
Fast thinking, slow thinking
The framework borrows the psychology split between System 1 and System 2. A general LLM, DeepSeek V4.1 Flash in the tests, is System 2. It understands the task, breaks it into goals, supplies values to type and reads results. A specially trained decision model is System 1. It picks the action on the page at each step in one forward pass of about a second, with no sampling and no generated text.
The division is strict. A goal says what result to reach and what limits to keep, not where the controls are. It also stops wherever judgment is needed: System 1 might halt at a list of results, and System 2 then reads the list and names the item. System 1 can’t open an address, type freely or click at coordinates, and any text it types has to come from System 2.
The cost of asking an LLM every step
Karpo’s argument is that mainstream agents, including browser-use, Stagehand, computer-use models and general coding agents, hand every step to a general model. A task takes twenty-odd steps, so cost and time grow with the step count. In the team’s evaluation, one general LLM turn takes about 3.4 seconds, reading roughly 19.5k input tokens and writing 476 output tokens.
The leaderboard data, 91 records updated November 18, 2026, suggests that paying more doesn’t buy accuracy. Every accuracy tier has a median time near 450 to 550 seconds and a median cost around $1. Accuracy has almost no rank correlation with either time or cost. Changing the framework around the same model barely moved results: deepseek-v4.1-flash in three frameworks scored 61% to 63%.
Per-step accuracy is the deeper problem. On KBB, Claude Fable 5.2 picks the right action on 82.1% of steps. With an average of 7.6 steps per instruction, only 40.8% of instructions are right at every step. The four general LLMs on the KBB board complete 38.1% to 41.0% of instructions, at 2.6 to 5.0 seconds and $0.009 to $0.083 per page decision.
How the numbers compare
The public leaderboard uses only 38 core tasks. There the browser agent averages 84% accuracy, 218 seconds and $0.096 over 8 runs. Karpo says none of the 91 records reaches 80% accuracy within 250 seconds and $0.15 per task.
- Twelve records exceed 80% accuracy, with theirs running from 84% to 92%.
- The fastest of them takes 223 seconds, and their median is 539 seconds.
- The cheapest costs $0.39 per task, about four times Karpo’s figure.
The two most accurate records are claude-fable-5-2 with Claude Code at 92%, 378 seconds and $1.24 per task, and gpt-6.2-astra with Codex at 89%, 223 seconds and $1.46. Seven Gemini 4 records carry introductory prices that double after November 30.
Built for messy pages
The framework includes rules drawn from tests on real sites worldwide. It reopens a page after a network error, waits for slow loads, avoids clicking when a page shifts mid-decision and won’t pay twice after a timeout. If it detects it is going in circles, it hands control back to the general LLM.
The KBB benchmark measures web action decisions only. It covers 584 real websites and 23,075 steps, with answers aligned to real human operations.
What to check before using it
The reported results come from Karpo’s own evaluation, so teams should run their own tasks before relying on the 89% figure. The framework code, decision weights and benchmark are all open, which makes that straightforward. The bigger question for the field is whether other agent builders adopt a small decision model for browser steps, since the numbers suggest the cost sits in the per-step loop rather than the choice of frontier model.
Source: karpo.ai