By Roman Aglaganov
AI agents that use a computer are only as good as the controls they are given. A new paper from Simular, posted on 20 October, argues that those controls should be designed by AI too, and shows that doing so lets a small open model compete with the biggest commercial ones.
The problem
The authors give two examples of how agents waste effort: typing a spreadsheet cell by cell when one line of code would do, or printing two thousand lines of a file to find one function. The shortcuts, and the knowledge of when to use them, are usually built by hand, which cannot keep up with new apps and models.
How Agent S4 works
Agent S4 adds a “meta-agent” that watches ordinary task agents work through practice tasks. It does not know whether they succeeded. It builds and revises a library of interfaces, small tools and notes on when to use each, from what it sees. A process modelled on evolution then keeps the libraries that perform best on validation tasks the meta-agent never sees, which favours changes that generalise.
The results
On OSWorld V2, a benchmark of real computer tasks, Qwen3.8-27B with an evolved library scored 21.3%, 3.8 times the same model with fixed interfaces. The paper puts that above GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%), at about 4.3% and 1.5% of their costs. On ScienceBoard, a test of scientific software, success rose from 41.4% to 58.6% with the same model. These are the authors’ own figures, and the code is open source.
The low absolute scores are a reminder of how hard these tasks still are: even the best system in the comparison fails most of them.
Why it matters
Most progress in agents has come from bigger models. This work suggests a second lever: better tools, built from experience, which an organisation could improve without training a new model. It also helps open models, which can run on a company’s own hardware.
