Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Exhausted man defeats AI model in world coding championship

arstechnica.com

Post reply on HN
Print view View original

41–50 of 61 posts

Re: Exhausted man defeats AI model in world coding championship

#41

Post by chiwilliams »

abound wrote: post #33

I'm guessing he didn't have access to any LLMs while competing, but I think a "centaur" approach probably would have outperformed both "only human" and "only LLM" competitors. Reading through the challenge, there's a lot of data modelling and test harness writing and ideating that an LLM could knock out fairly quickly, but would take even a competitive coder some time to write (even if just limited by typing speed).…

He did use a little autocomplete apparently, but used [Vscode](https://x.com/jacob_posel/status/1945585787690738051).

And it's not against the rules to use LLMs apparently in the competition. (https://atcoder.jp/posts/1495). I'd be curious what other competitors used.

Re: Exhausted man defeats AI model in world coding championship

#42

Post by xienze »

reducesuffering wrote: post #32
atleastoptimal wrote: post #31

Remember this is the worst AI will ever be from here on out. Models are only going to get better, faster, cheaper, more accessible and more easily deployable. I think people need to realize that just because an AI model fails at one point, or some certain architecture has common failure modes, that billions of dollars are poured into correcting those failures and improving in every economically viable domain. Two yea…

Exactly. The inability of people to extrapolate towards the future and foresee second-order effects is astounding. We've seen this in climate change and we've just seen this in COVID. The ones with foresight are warning about the massive upheaval coming. It's time for people to shake away their preconceived notions, look at the situation with fresh eyes, and deeply think about what the technology diff from 5 years ag…

> Exactly. The inability of people to extrapolate towards the future and foresee second-order effects is astounding.

On a related note, many people also assume that just because something has been trending exponential that it will _continue_ to do so...

Re: Exhausted man defeats AI model in world coding championship

#43

Post by tyre »

ChrisMarshallNY wrote: post #27
coldtea wrote: post #24

Earlier quoted context omitted.

Well, millions did, that's why it's a classic!

I suspect it may not be taught, anymore, though. I seem to encounter cultural milestones, that are no longer there, every day.

I learned it in elementary school in the late 90s

Re: Exhausted man defeats AI model in world coding championship

#44

Post by chiwilliams »

flanbiscuit wrote: post #25

so many things First, there's a world coding championship?! Of course there is. There's a competition for anything these days. Why is he exhausted? > The 10-hour marathon left him "completely exhausted." > ... noting he had little sleep while competing in several competitions across three days. "I'm completely exhausted. ... I'm barely alive." oh! That's a lot. > beating an advanced AI model from OpenAI ... > On Wedn…

He's retired, so I'm guessing more about the clout. Or even just "love of the game"? He had a fairly popular tweet thread a couple years back where he wrote out 80 tips for competitive programming -- that feels less likely to be clout based

Re: Exhausted man defeats AI model in world coding championship

#45

Post by abound »

chiwilliams wrote: post #41
abound wrote: post #33

I'm guessing he didn't have access to any LLMs while competing, but I think a "centaur" approach probably would have outperformed both "only human" and "only LLM" competitors. Reading through the challenge, there's a lot of data modelling and test harness writing and ideating that an LLM could knock out fairly quickly, but would take even a competitive coder some time to write (even if just limited by typing speed).…

He did use a little autocomplete apparently, but used [Vscode]( https://x.com/jacob_posel/status/1945585787690738051 ). And it's not against the rules to use LLMs apparently in the competition. ( https://atcoder.jp/posts/1495 ). I'd be curious what other competitors used.

Interesting, thanks for the links! I had read this part of the article:

> All competitors, including OpenAI, were limited to identical hardware provided by AtCoder, ensuring a level playing field between human and AI contestants.

And assumed that meant a pretty restricted (and LLM-free) environment. I think their policy is pretty pragmatic.

Re: Exhausted man defeats AI model in world coding championship

#46

Post by stevage »

atleastoptimal wrote: post #31

Remember this is the worst AI will ever be from here on out. Models are only going to get better, faster, cheaper, more accessible and more easily deployable. I think people need to realize that just because an AI model fails at one point, or some certain architecture has common failure modes, that billions of dollars are poured into correcting those failures and improving in every economically viable domain. Two yea…

> AGI will arrive.

This does not follow. Your argument, set in the 1950s, would be that cars keep getting faster, therefore they will reach light speed.

Re: Exhausted man defeats AI model in world coding championship

#47

Post by _puk »

atleastoptimal wrote: post #31

Remember this is the worst AI will ever be from here on out. Models are only going to get better, faster, cheaper, more accessible and more easily deployable. I think people need to realize that just because an AI model fails at one point, or some certain architecture has common failure modes, that billions of dollars are poured into correcting those failures and improving in every economically viable domain. Two yea…

Haven't they already started to regress?

I'm bullish on specific areas improving (I'm sure you could selectively train an LLM on the latest Angular version to replace the majority of front-end devs given enough time and money, it's a limited problem space and a strongly opinionated framework after all), but for the most part enshittification is already starting to happen with the general models.

Nowadays even ChatGPT doesn't bother to even refer to the original question posed after a few responses, so you're left summarising a conversation and starting a new context to get anywhere.

So, yeah, I think we're very much into finding the equilibrium now. Cost vs scale. Exponential improvements won't be in the general LLMs.

Happy to be wrong on this one..

Re: Exhausted man defeats AI model in world coding championship

#48

Post by ASalazarMX »

ChrisMarshallNY wrote: post #14
esseph wrote: post #10

Earlier quoted context omitted.

The article mentions it...

Yup. I was talking about how they taught it to us, in school. It actually had an emotional place in my heart. For some reason, I found the story compelling.

I found it sad, because he dies at the end to prove he could beat the machine once, but the machine could keep producing its lesser output every day after his death. He gave it all for a Pyrrhic victory.
Post reply on HN