Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind and OpenAI win gold at ICPC

codeforces.com

Post reply on HN
Print view View original

31–40 of 255 posts

Re: DeepMind and OpenAI win gold at ICPC

#31

Post by birktj »

XenophileJKO wrote: post #19
birktj wrote: post #5

They apparently managed gold in the IOI as well. A result that was extremely surprising for me and causes me to rethink a lot of assumptions I have about current LLMs. Unfortunately there was very little transparency on how they managed those results and the only source was a Twitter post. I want to know if there was any third party oversight, what kind of compute they used, how much power what kind of models and how…

So outside of human intervention, I don't think the specifics really matter. What this means is that it is possible and that this capability will in time be commoditized. This is helpful in framing the conversation, especially with "skeptics" of what these models are capable of.

To a certain extent I agree. But as far as I know I cannot go to chatgpt.com and paste the newest ICPC problems and get full solutions. And there is no information about what they do differently. For a competition like the ICPC, which is academic in its nature, I think it is very unfortunate to setup a seperate AI track like this without publishing clear public information about what that actually entails. And have clear requirements for these AI companies to publish their methology. I know it is a nice source of sponsorships for them, but the ICPC should afford to stand up a bit for academic integrity.

Without any of this I can't even know for sure if there was any human intervention. I don't really think so, but as I mentioned the financial pressure to perform well is extreme so I can totally see that happening. Maybe ICPC did have some oversight, but please write a bit about it then.

If you assume no human intervention then all of this is of course irrelevant if you only care about the capabilities that exist. But still the implications of a general model performing at this level vs something more like a chess model trained specifically on competitive programming are of course different, even if the gap may close in the future. And how much compute/power was used, are we talking hundreds of kWhs? And does that just means larger models than normally or intelligent bruteforcing through a huge solutionspace? If so, then it is not clear how much they will be able to scale down the compute usage while keeping the performance at the same level

Re: DeepMind and OpenAI win gold at ICPC

#32

Post by andy12_ »

jaggs wrote: post #13

I think it's becoming clear that these mega AI corps are juggling with their models at inference time to produce unrealistically good results. By that it seems that they're just cranking up the compute beyond reasonable levels in order to gain PR points against each other. The fact is most ordinary mortals never get access to a fraction of that kind of power, which explains the commonly reported issues with AI models…

Models drop in price x10 each year. Us, common folk, getting access to these kinds of models is just a matter of time.

Re: DeepMind and OpenAI win gold at ICPC

#33

Post by jaggs »

andy12_ wrote: post #32
jaggs wrote: post #13

I think it's becoming clear that these mega AI corps are juggling with their models at inference time to produce unrealistically good results. By that it seems that they're just cranking up the compute beyond reasonable levels in order to gain PR points against each other. The fact is most ordinary mortals never get access to a fraction of that kind of power, which explains the commonly reported issues with AI models…

Models drop in price x10 each year. Us, common folk, getting access to these kinds of models is just a matter of time.

Is that true though? Having to pay some $200 a month for a max account of whatever kind doesn't seem to be cheaper to me at all?

Re: DeepMind and OpenAI win gold at ICPC

#34

Post by bgwalter »

tgma wrote: post #27
ferguess_k wrote: post #25

I think in the future information will be more walled -- because AI companies are not paying anyone for that piece of information, and I encourage everyone to put their knowledge on their own website, and for each page, put up a few urls that humans won't be able to find (but can still click if he knows where to find), but can be crawled by AI, which link to pages containing falsified information (such as, oh the inf…

Why the AI hate? How is it different from sharing your knowledge with another individual or writing a book to share it? > AI companies are not paying anyone for that piece of information So? For the vast majority of human existence, paying for content was not a thing, just like paying for air isn't. The copyright model you are used to may just be too forced. Many countries have no moral qualms about "pirating" Window…

Companies valued at $300 billion or more are not another individual and people are not "sharing" their works. The companies are stealing them.

For the majority of interesting output people have paid for art, music, software, journalism. But you know that already and are justifying the industry that pays your bills.

Re: DeepMind and OpenAI win gold at ICPC

#35

Post by bgwalter »

A database is good at leetcode, who would have thought. Give humans a database and they'll outperform your "AI" (which probably uses an extraordinary amount of graphics cards and electricity).

It is an idiotic benchmark, in line with the rest of the "AI" propaganda.

Re: DeepMind and OpenAI win gold at ICPC

#36

Post by scarmig »

jaggs wrote: post #33
andy12_ wrote: post #32

Earlier quoted context omitted.

Models drop in price x10 each year. Us, common folk, getting access to these kinds of models is just a matter of time.

Is that true though? Having to pay some $200 a month for a max account of whatever kind doesn't seem to be cheaper to me at all?

$200/month for an LLM with the capability to fully automate my job is extremely cheap. Of course, even with a high thinking budget we don't have that yet, but if we see it at any cost in 2026, I'll be expecting to be forced into retirement by 2030.

Re: DeepMind and OpenAI win gold at ICPC

#37

Post by andy12_ »

jaggs wrote: post #33
andy12_ wrote: post #32

Earlier quoted context omitted.

Models drop in price x10 each year. Us, common folk, getting access to these kinds of models is just a matter of time.

Is that true though? Having to pay some $200 a month for a max account of whatever kind doesn't seem to be cheaper to me at all?

When I say 10 times cheaper, I mean when comparing models of the same capabilities. The kind of performance you get now for a 200$ subscription, a year ago probably would have costed 2000$.

Re: DeepMind and OpenAI win gold at ICPC

#38

Post by birktj »

bgwalter wrote: post #34
tgma wrote: post #27

Earlier quoted context omitted.

Why the AI hate? How is it different from sharing your knowledge with another individual or writing a book to share it? > AI companies are not paying anyone for that piece of information So? For the vast majority of human existence, paying for content was not a thing, just like paying for air isn't. The copyright model you are used to may just be too forced. Many countries have no moral qualms about "pirating" Window…

Companies valued at $300 billion or more are not another individual and people are not "sharing" their works. The companies are stealing them. For the majority of interesting output people have paid for art, music, software, journalism. But you know that already and are justifying the industry that pays your bills.

Absolutely, I am sceptical of AI omin many ways, but primarily it is about the AI companies and my lack of trust in them. I find it unfortunate that all of the clearly brilliant engineers working at these companies are to preoccupied with always chasing newer and better model trying to reach the dream of AGI do not stop and ask themselves: who are they working for? What happens if they eventually manage to create a model that can replace most or even all of human computer work?

Why whould anyone think that these companies will contribute to the good of humanity when they are even bigger and more powerful, when they seem to care so little now?

Re: DeepMind and OpenAI win gold at ICPC

#39

Post by tgma »

bgwalter wrote: post #34
tgma wrote: post #27

Earlier quoted context omitted.

Why the AI hate? How is it different from sharing your knowledge with another individual or writing a book to share it? > AI companies are not paying anyone for that piece of information So? For the vast majority of human existence, paying for content was not a thing, just like paying for air isn't. The copyright model you are used to may just be too forced. Many countries have no moral qualms about "pirating" Window…

Companies valued at $300 billion or more are not another individual and people are not "sharing" their works. The companies are stealing them. For the majority of interesting output people have paid for art, music, software, journalism. But you know that already and are justifying the industry that pays your bills.

> valued at $300 billion

Irrelevant really. Invoking this in the argument shows the basis is jealousy. They are clearly valued as such not because they collected all the data and stored in some database. Your local library is not worth 300 billion.

> For the majority of interesting output people have paid for art, music, software, journalism

Absolutely and demonstrably false. Music and art predate Copyright by hundreds if not thousands of years.

> But you know that already and are justifying the industry that pays your bills.

Huh, ad hominem much? I find it rich that the whole premise of your argument was some "art, music, software, journalist" was entitled to some payment, but suddenly it is a problem when "my industry" (somehow you assume I work in AI) is getting paid?

Re: DeepMind and OpenAI win gold at ICPC

#40

Post by jaggs »

andy12_ wrote: post #37
jaggs wrote: post #33

Earlier quoted context omitted.

Is that true though? Having to pay some $200 a month for a max account of whatever kind doesn't seem to be cheaper to me at all?

When I say 10 times cheaper, I mean when comparing models of the same capabilities. The kind of performance you get now for a 200$ subscription, a year ago probably would have costed 2000$.

I understand what you're saying. However I'm not sure it's that germane when we're talking about whether or not the current $200 subscription fee is actually delivering value for money, or whether AI giants are manipulating performance to gain marketing points.
Post reply on HN