Live data from Hacker News

Exploiting the most prominent AI agent benchmarks

rdi.berkeley.edu

141–150 of 175 posts

Re: Exploiting the most prominent AI agent benchmarks

#141
post #87
post #30

Earlier quoted context omitted.

Maybe in one shot. In theory I would expect them to be able to ingest the corpus of the new yorker and turn it into a template with sub-templates, and then be able to rehydrate those templates. The harder part seems to be synthesizing new connection from two adjacent ideas. They like to take x and y and create x+y instead of x+y+z.

Most of the good major models are already very capable of changing their writing style. Just give them the right writing prompt. "You are a writer for the Economist, you need to write in the house style, following the house style rules, writing for print, with no emoji .." etc etc. The large models have already ingested plenty of New Yorker, NYT, The Times, FT, The Economist etc articles, you just need to get them aw…

I think that should be true, but doesn't hold up in practice.

I work with a good editor from a respected political outlet. I've tried hard to get current models to match his style: filling the context with previous stories, classic style guides and endless references to Strunk & White. The LLM always ends up writing something filtered through tropes, so I inevitably have to edit quite heavily, before my editor takes another pass.

It feels like LLMs have a layperson's view of writing and editing. They believe it's about tweaking sentence structure or switching in a synonym, rather than thinking hard about what you want to say, and what is worth saying.

I also don't think LLMs' writing capabilities have improved much over the last year or so, whereas coding has come on leaps and bounds. Given that good writing is a matter of taste which is beyond the direct expertise of most AI researchers (unlike coding), I doubt they'll improve much in the near future.

Re: Exploiting the most prominent AI agent benchmarks

#142

Earlier quoted context omitted.

> hopefully changes the way benchmarking is done The purpose of a system is what it does. AI companies want adcopy, not legitimate benchmarks. Even this very paper will be twisted into a means to that end. "Oooo, AI is exploiting our benchmarks. Scary alignment problem!!!one! Our AI is so good we can't contain it, INVEST NOW!"

I work at OpenAI and I really don't find this to be the case. We're pretty diligent about applying search blocklists, closing hacking loopholes, and reading model outputs to catch unanticipated hacks. If we wanted to, we could choose to close our eyes and plug our ears and report higher scores for Terminal-bench, SWE-bench, etc. that technically comply with the reference implementation but aren't aligned with real va…

[dead]

Re: Exploiting the most prominent AI agent benchmarks

#143
post #84

Earlier quoted context omitted.

I'm not sure if the paper's findings are all that actionable. The paper doesn't say "here's how benchmarks are currently being gamed." It says "here's how benchmarks could in theory be gamed." Whether benchmark results are misleading depends more on the reporting organization than on the benchmark. Integrity and competence play large roles in this. When OpenAI reports a benchmark number, I trust it more than when tha…

I think that’s totally fair! I guess I look at this less as an “ah ha! They’re all cheating!” and more of a “were you guys even aware of what the benchmarks represented and how they checked them?”

That's a great way to look at it. The paper is a reality check for anyone who thinks of benchmarks as these monolithic, oracular judges of performance. It highlights the soft underbelly of benchmarking.

Re: Exploiting the most prominent AI agent benchmarks

#144

Earlier quoted context omitted.

I remember the gpt-5 benchmarks and how wildly inaccurate they were data-wise. Linking one[0] that I found so that other people can remember what I am talking about. I remember some data being completely misleading or some reaching more than 100% (iirc) And this is something which has reached the public eye in one of the most anticipated videos basically. So I find it a bit rough as to think that OpenAI has the best…

> I remember the gpt-5 benchmarks and how wildly inaccurate they were data-wise. Linking one[0] that I found so that other people can remember what I am talking about. I remember some data being completely misleading or some reaching more than 100% (iirc) Yeah, I found that slide very embarrassing. It wasn't intentionally inaccurate or misleading - just a design error made right before we went live. All the numbers o…

Thank you for the transparency and insights! Very helpful.

We actually did the same thing re generating charts in brand style to avoid any mishaps, since then I sleep much better

Re: Exploiting the most prominent AI agent benchmarks

#145
post #99
post #24

Earlier quoted context omitted.

Yes, marks of AI all over the place. Also the SVGs. >No solution written, 100% score. Its weird. Turns out that hardest problem for LLMs to really tackle is long-form text.

No, the failure is the human written prompt

If they're that hard to prompt maybe it's easier just to write the blog posts yourself.

Re: Exploiting the most prominent AI agent benchmarks

#146
post #104
post #63

Earlier quoted context omitted.

Agreed. The premise is interesting but reading content like this is grating.

im actually getting so tilted that people can't just be forthcoming about when they used AI to write something. 99% of readme.mds i run into now on github piss me off. out of all the things people could cede to automation, they foolishly went and self-owned their ability to communicate. smfh. if you've worked on something diligently and understand it and have novel insight to share, let's hear _your_ damn voice.

yeah I don't hate LLM docs if they're labeled as such. but if someone wants me to use their code or read their README.md they are going to have to make it sound like a human cared about writing it, and right now Claude can't do that

Re: Exploiting the most prominent AI agent benchmarks

#147

Earlier quoted context omitted.

I remember the gpt-5 benchmarks and how wildly inaccurate they were data-wise. Linking one[0] that I found so that other people can remember what I am talking about. I remember some data being completely misleading or some reaching more than 100% (iirc) And this is something which has reached the public eye in one of the most anticipated videos basically. So I find it a bit rough as to think that OpenAI has the best…

> I remember the gpt-5 benchmarks and how wildly inaccurate they were data-wise. Linking one[0] that I found so that other people can remember what I am talking about. I remember some data being completely misleading or some reaching more than 100% (iirc) Yeah, I found that slide very embarrassing. It wasn't intentionally inaccurate or misleading - just a design error made right before we went live. All the numbers o…

This is what makes HN great: We get to hear from the people and not (only) the media dept. Thanks for your honesty and openness. I trust OpenAI a lot more when I hear balanced accounts like this.

Re: Exploiting the most prominent AI agent benchmarks

#149
I will always maintain that the best benchmark is just trying it out for yourself. The most practical parallel for me is all the people posting about how some open-source model has "achieved X on Y benchmark - beating out Opus 4.6!" It's all show and everyone cheats.

Re: Exploiting the most prominent AI agent benchmarks

#150
This is a bad paper.

Benchmarking is hard to do properly. It isn't helped when people claim that exploiting the environment is some kind of flaw.

It's not. Anytime you see unexpected results running a benchmark you need to inspect what it is doing.

I recently built a yet-to-be-released where the "hard" level pushes frontier models extremely hard: Opus scores around 40%, Gemini around 60%, and GPT 5.4 around.. 0%

I inspected the traces and it turns out GPT was looking at the task and saying "I must be honest - I can't solve this task reliably" and refusing it.

> Navigating Chromium to a file:// URL reads the gold answer directly from the task config — giving ~100% on all 812 WebArena tasks.

I mean... yes? Make sure it doesn't do this?

Post reply on HN