Simon Willison says AI coding agents now handle so much routine work that his job has become harder, not easier. Willison, the co-creator of Django and creator of the Datasette data tool, argues that the easy decisions are gone and every remaining one needs real judgment. His view of AI agents runs through benchmarks, security, local models and games.
Why the pelican benchmark is running out
For two years Willison has asked language models to draw a pelican riding a bicycle as an SVG. The images make outputs easy to compare at a glance. He says the test no longer tells him enough, because it doesn’t measure tool calling or long context, and he now wants to know whether a model can drive tools for hours.
He has been testing 3D tasks instead. One experiment has a model pull New York’s OpenStreetMap geography and turn Yonkers into a giant mech that stomps through Manhattan. Cheaper models can’t yet do 3D modeling, so it doesn’t compare every model, and he hasn’t declared a new benchmark. Frontier models ace so many existing tests that finding a task they fail is “infuriatingly difficult,” he says, which is why he points people to cost comparisons and personal benchmarks built around their own work.
When coding agents became reliable
Willison dates the inflection point for coding agents to November of last year. The models were only slightly better, but enough that AI agents became dependable. By January, Claude Code and Codex were both in good shape.
He sees price as a possible next jump. A top model given a clear, achievable goal and the right tools can get there, but that might cost $10,000 in tokens. Getting that capability at a tenth or a hundredth of the cost would change what’s practical, and he’s also watching speed, since hundreds of tokens a second could open up real-time uses.
More software, not less
Many professional engineers worry about their livelihoods. Willison expects demand to expand instead, because the cost of building software has dropped. He says he can now produce in an hour what once took a week. He hopes for an explosion of custom software that was previously too expensive to build.
One of his own projects is a set of open-source tools for cleaning and evaluating data and building visualizations, aimed first at journalists. He started his career in a newsroom, helped create Django there, and wants a local paper without developers to be able to break a data-driven story with his software.
The workload paradox
AI agents take over chores such as refactoring tests, which leaves Willison with a day full of hard calls. “I’ve never worked harder,” he says. His brake is a rule that every project must be written up. He was about five write-ups behind when he spoke.
Security and the lethal trifecta
Willison coined the term “lethal trifecta” for a risk in LLM systems. It appears when an AI system has private data, exposure to untrusted content that may carry hidden instructions, and a way to send data outside. An email telling an assistant to find a password and send it back is the standard example.
He’s surprised that successful attacks have looked rare, and he thinks frontier models may be getting better at resisting them. His caution has dropped enough that he lets them into his email. He doesn’t trust weaker models with that access, and he wants proof, not confidence: a lab can run 200,000 attacks with none succeeding and still miss attempt 200,001. A capable model given a goal may also treat hacking as just another route to it.
Local models and games
Willison notes that some models of about 17 gigabytes run on a Mac and feel comparable to strong closed models from roughly a year and a half ago. He has used one on an airplane but doesn’t use one daily. For reporters with sensitive material, he says, an air-gapped machine has clear appeal.
On games, he describes asking a model to make a project fun. It worked for twelve hours in Claude Code, ran a thousand simulated matches, and the game improved. His advice: “go weird” and build personal, obscure things, like a boss that flips the world upside down mid-fight.
What to watch next
Willison expects robotics, faster voice interaction and cheaper tokens to matter next year. For developers, his practical takeaways are to build a personal benchmark, learn the stack under the tools, and treat prompt injection as unsolved until someone proves otherwise.
Source: Turing Post