- Alibaba and several Asian universities built Co-Design Bench, a coding-agent benchmark drawn from real user change requests that previously tripped up AI agents.
- GPT-6 Astra, the top performer among ten tested models, meets about 70% of a task’s requirements on average but is fully correct on only 26.4% of tasks.
- Older benchmarks like WebCompass-edit now score agentic coding models above 99%, a gap of more than 35 percentage points versus Co-Design Bench.
- Claude Opus 5 and Qwen3.8-Max rank just behind GPT-6 Astra; Grok 4.6 placed last among the ten models tested.
What Happened
Researchers from Alibaba and several Asian universities released Co-Design Bench on November 8, 2026, a benchmark that tests coding agents on real user requests to rework existing web apps, including tasks that supply images, audio, video, or documents as input.
Why It Matters
The authors built the new test because existing coding benchmarks have become saturated: when models can run, check, and fix their own code, scores on tests like WebCompass-edit climb above 99%, leaving little room to compare models. Co-Design Bench instead draws on sessions where users had to rephrase requests or roll back changes.
Technical Details
Grading is strict: one AI judge tests the finished app in a browser, and a second judge re-checks every positive verdict for mockups or hardcoded results. GPT-6 Astra leads at 26.4% full completion, with Claude Opus 5 and Qwen3.8-Max next and Grok 4.6 last. On larger, full-scale overhauls, the smaller Qwen3.8-27B climbs to second place, though the authors don’t draw a general conclusion from that result.
Who’s Affected
Every model’s score drops further as a task combines more types of changes — layout, design, interaction, and content together, which the authors note is the norm in real projects. That leaves developers relying on these agents needing to manually finish or correct the large majority of real-world requests.
What’s Next
The researchers have published the benchmark and paper for other labs to test against, and expect scores to keep lagging well behind saturated legacy benchmarks until models handle multi-type change requests more reliably.
