Top AI models fail at >96% of tasks

#1

Post by codexon » Sat, Feb 07, 2026, 9:20 PM UTC

Top AI models fail at >96% of tasks
zdnet.com

Re: Top AI models fail at >96% of tasks

#2

Post by codexon » Sat, Feb 07, 2026, 9:20 PM UTC

This paper creates a new benchmark comprised of real remote work tasks sourced from the remote working website Upwork. The best commercial LLMs like Opus, GPT, Gemini, and Grok were tested.

Models released a few days ago, Opus 4.6 and GPT 5.3, haven't been tested yet, but given the performance on other micro-benchmarks, they will probably not be much different on this benchmark.

Re: Top AI models fail at >96% of tasks

#3

Post by Venn1 » Sat, Feb 07, 2026, 9:33 PM UTC

ChatGPT: when you want spellcheck to argue with you.

Re: Top AI models fail at >96% of tasks

#4

Post by kolinko » Sat, Feb 07, 2026, 11:13 PM UTC

codexon wrote: post #2

This paper creates a new benchmark comprised of real remote work tasks sourced from the remote working website Upwork. The best commercial LLMs like Opus, GPT, Gemini, and Grok were tested. Models released a few days ago, Opus 4.6 and GPT 5.3, haven't been tested yet, but given the performance on other micro-benchmarks, they will probably not be much different on this benchmark.

They didn't test Opus at all, only Sonnet.

One of the tasks was "Build an interactive dashboard for exploring data from the World Happiness Report." -- I can't imagine how Opus4.5 could've failed that.

Re: Top AI models fail at >96% of tasks

#5

Post by zb3 » Sun, Feb 08, 2026, 12:00 AM UTC

You think they don't? You think AI can replace programmers, today?

Then go ahead and use AI to fix this: https://gitlab.gnome.org/GNOME/mutter/-/issues/4051

Re: Top AI models fail at >96% of tasks

#6

Post by [deleted] » Sun, Feb 08, 2026, 1:06 AM UTC

[deleted]

Re: Top AI models fail at >96% of tasks

#7

Post by tessitore » Sun, Feb 08, 2026, 2:08 AM UTC

This post really should be edited to say 96% of tasks posted on Upwork. Since we would all expect that to happen.

Re: Top AI models fail at >96% of tasks

#8

Post by [deleted] » Sun, Feb 08, 2026, 1:37 PM UTC

[deleted]

Re: Top AI models fail at >96% of tasks

#9

Post by scotty79 » Sun, Feb 08, 2026, 1:40 PM UTC

Kinda sus that least known model did best and none of the more recent models were tested. Capabilities grow very fast. So things that now routinely succeed rarely ever succeeded even half a year ago.

Re: Top AI models fail at >96% of tasks

#10

Post by stoneforger » Sun, Feb 08, 2026, 3:29 PM UTC

zb3 wrote: post #5

You think they don't? You think AI can replace programmers, today? Then go ahead and use AI to fix this: https://gitlab.gnome.org/GNOME/mutter/-/issues/4051

Rewrite it in react it will.