Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI O3 breakthrough high score on ARC-AGI-PUB

arcprize.org

Post reply on HN
Print view View original

131–140 of 1001 posts

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#131

Post by QuantumGood »

modeless wrote: post #99
refulgentis wrote: post #71

Earlier quoted context omitted.

This emphasizes persons and a self-conceived victory narrative over the ground truth. Models have regularly made progress on it, this is not new with the o-series. Doing astoundingly well on it, and having a mutually shared PR interest with OpenAI in this instance, doesn't mean a pile of visual puzzles is actually AGI or some well thought out and designed benchmark of True Intelligence(tm). It's one type of visual pu…

Solving ARC doesn't mean we have AGI. Also o3 presumably isn't doing program synthesis, seemingly proving Francois wrong on that front. (Not sure I believe the speculation about o3's internals in the link.) What I'm saying is the fact that as models are getting better at reasoning they are also scoring better on ARC proves that it is measuring something relating to reasoning. And nobody else has come up with a compar…

Gaming the benchmarks usually needs to be considered first when evaluating new results.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#132

Post by getpost »

vessenes wrote: post #112

[flagged]

> most of the people training these next-gen AIs are neurodiverse and we are training the AI in our own image

Do you have any evidence to support that? It would be fascinating if the field is primarly advancing due to a unique constellation of traits contributed by individuals who, in the past, may not have collaborated so effectively.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#133

Post by whynotminot »

rvz wrote: post #73

Great results. However, let's all just admit it. It has well replaced journalists, artists and on its way to replace nearly both junior and senior engineers. The ultimate intention of "AGI" is that it is going to replace tens of millions of jobs. That is it and you know it. It will only accelerate and we need to stop pretending and coping. Instead lets discuss solutions for those lost jobs. So what is the replacement…

When none of us have jobs or income, there will be no ability for us to buy products. And then no reason for companies to buy ads to sell products to people who don’t have money. Without ad money (or the potential of future ad money), the people pushing the bounds of AGI into work replacement will lose the very income streams powering this research and their valuations.

Ford didn’t support a 40 hour work week out of the kindness of his heart. He wanted his workers to have time off for buying things (like his cars).

I wonder if our AGI industrialist overlords will do something similar for revenue sharing or UBI.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#134

Post by wruza »

hansonkd wrote: post #34
zebomon wrote: post #19

My initial impression: it's very impressive and very exciting. My skeptical impression: it's complete hubris to conflate ARC or any benchmark with truly general intelligence. I know my skepticism here is identical to moving goalposts. More and more I am shifting my personal understanding of general intelligence as a phenomenon we will only ever be able to identify with the benefit of substantial retrospect. As it is…

It doesn't need to be general intelligence or perfectly map to human intelligence. All it needs to be is useful. Reading constant comments about LLMs can't be general intelligence or lack reasoning etc, to me seems like people witnessing the airplane and complaining that it isn't "real flying" because it isn't a bird flapping its wings (a large portion of the population held that point of view back then). It doesn't…

And look at the airplanes, they really can’t just land on a mountain slope or a tree without heavy maintenance afterwards. Those people weren’t all stupid, they questioned the promise of flying servicemen delivering mail or milk to their window and flying on a personal aircar to their workplace. Just like todays promises about whatever the CEOs telltales are. Imagining bullshit isn’t unique to this century.

Aerospace is still a highly regulated area that requires training and responsibility. If parallels can be drawn here, they don’t look so cool for a regular guy.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#135

Post by CliveBloomers »

Another meaningless benchmark, another month—it’s like clockwork at this point. No one’s going to remember this in a month; it’s just noise. The real test? It’s not in these flashy metrics or minor improvements. The only thing that actually matters is how fast it can wipe out the layers of middle management and all those pointless, bureaucratic jobs that add zero value.

That’s the true litmus test. Everything else? It’s just fine-tuning weights, playing around the edges. Until it starts cutting through the fat and reshaping how organizations really operate, all of this is just more of the same.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#136

Post by 6gvONxR4sf7o »

I'm glad these stats show a better estimate of human ability than just the average mturker. The graph here has the average mturker performance as well as a STEM grad measurement. Stuff like that is why we're always feeling weird that these things supposedly outperform humans while still sucking. I'm glad to see 'human performance' benchmarked with more variety (attention, time, education, etc).

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#137

Post by foobarqux »

razodactyl wrote: post #4

Great. Now we have to think of a new way to move the goalposts.

This is just as silly as claiming that people "moved the goalposts" when a computer beat Kasparov at chess to claim that it wasn't AGI: it wasn't a good test and some people only realize this after the computer beat Kasparov but couldn't do much else. In this case the ARC maintainers specifically have stated that this is a necessary but not sufficient test of AGI (I personally think it is neither).

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#138

Post by intelVISA »

surgical_fire wrote: post #98
hansonkd wrote: post #34

Earlier quoted context omitted.

It doesn't need to be general intelligence or perfectly map to human intelligence. All it needs to be is useful. Reading constant comments about LLMs can't be general intelligence or lack reasoning etc, to me seems like people witnessing the airplane and complaining that it isn't "real flying" because it isn't a bird flapping its wings (a large portion of the population held that point of view back then). It doesn't…

> to me seems like people witnessing the airplane and complaining that it isn't "real flying" because it isn't a bird flapping its wings To me it is more like there is someone jumping on a pogo ball while flapping their arms and saying that they are flying whenever they hop off the ground. Skeptics say that they are not really flying, while adherents say that "with current pogo ball advancements, they will be flying…

Between skeptics and adherents who is more easily able to extract VC money for vaporware? If you limit yourself to 'the facts' you're leaving tons of $$ on the table...

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#140

Post by antirez »

notfish wrote: post #126
antirez wrote: post #118

Earlier quoted context omitted.

NNs are not algorithms.

An algorithm is “a process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer” How does a giant pile of linear algebra not meet that definition?

It's not made of "steps", it's an almost continuous function to its inputs. And a function is not an algorithm: it is not an object made of conditions, jumps, terminations, ... Obviously it has computation capabilities and is Turing-complete, but is the opposite of an algorithm.
Post reply on HN