Computer Use Agents: Screenshots vs Accessibility Tree
Greptile tested seven models on 172 OSWorld tasks. Accessibility trees and element IDs helped weak models a lot, while the strongest ones barely moved.
Greptile tested seven models on 172 OSWorld tasks. Accessibility trees and element IDs helped weak models a lot, while the strongest ones barely moved.