Live data from Hacker News

OpenAI O3 breakthrough high score on ARC-AGI-PUB

arcprize.org

631–640 of 1001 posts

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#631
post #569

> You'll know AGI is here when the exercise of creating tasks that are easy for regular humans but hard for AI becomes simply impossible. You'll know AGI is here when traditional captchas stop being a thing due to their lack of usefulness.

(Shrug) AI has been better than humans at solving CAPTCHAs for a LONG time. As the sibling points out, they're just a waste of time and electricity at this point.

Ironically, they are used as free labor to label image sets for ai to be trained on.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#632
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

The future belongs to those who believe there will be one.

That is: If you don't believe there will be a future, you give up on trying to make one. That means that any kind of future that takes persistent work becomes unavailable to you.

If you do believe that there will be a future, you keep working. That doesn't guarantee there will be a future. But not working pretty much guarantees that there won't be one, at least not one worth having.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#633
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

I feel like more likely a lot of jobs (CS and otherwise ) are going to go the way of photography. Your average person now can take amazing photos but you’re still going to use a photographer when it really matters and they will use similar but more professional tools to be more productive. Low end bad photographers probably aren’t doing great but photography is not dead. In fact the opposite is true, there are millio…

It doesn't comfort me when people say jobs will "go the way of photography". Many choose to go into STEM fields for financial stability and opportunity. Many do not choose the arts because of the opposite. You can point out outlier exceptions and celebrities, but I find it hard to believe that the rare cases where "it really matters" can sustain the other 90% who need income.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#634
Very convenient for OpenAI to run those errands with bunch of misanthropes trying to repaint a simulacrum. To use AGI here's makes me want to sponsor pile of distress pills so people think things really over before going into another mania Episode. People need seriously take a step back, if that's AGI then my cat has surpassed it's cognitive acting twice.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#635
post #620

I'm 22 and have no clue what I'm meant to do in a world where this is a thing. I'm moving to a semi rural, outdoorsy area where they teach data science and marine science and I can enjoy my days hiking, and the march of technology is a little slower. I know this will disrupt so much of our way of life, so I'm chasing what fun innocent years are left before things change dramatically.

I agree completely. This is a fundamentally different change than the ones that came before. Calculators, assemblers, higher level languages, none of these actually removed the _reasoning_ the engineer has to do, they just provide abstractions that make this reasoning easier. What reason is there to believe LLMs will remain "assistants" instead of becoming outright replacements? If LLMs can do the reasoning all the w…

What was it that initially inspired you to learn to code? Was it robots, video games, design, etc... Whatever that was, creating the pinnacle of it is what your future will be.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#636
post #19

My initial impression: it's very impressive and very exciting. My skeptical impression: it's complete hubris to conflate ARC or any benchmark with truly general intelligence. I know my skepticism here is identical to moving goalposts. More and more I am shifting my personal understanding of general intelligence as a phenomenon we will only ever be able to identify with the benefit of substantial retrospect. As it is…

how about a extra large dose of your skepticism. is true intelligence really a thing and not just a vague human construct that tries to point out the mysterious unquantifiable combination of human behaviors? humans clearly dont know what intelligence is unambiguously. theres also no divinely ordained objective dictionary that one can point at to reference what true intelligence is. a deep reflection of trying to patt…

My guess as an amateur neuroscientist is that what we call intelligence is just a 'measurement' of problem solving ability in different domains. Can be emotional, spatial, motor, reasoning, etc etc.

There is no special sauce in our brain. And we know how much compute there is in our brain– So we can roughly estimate when we'll hit that with these 'LLMs'.

Language is important in a human brain development as well. Kids who grow up deaf grow up vastly less intelligent unless they learn sign language. Language allow us to process complex concepts that our brain can learn to solve, without having to be in those complex environments.

So in hindsight, it's easy to see why it took a language model to be able to solve general tasks and other types deep learning networks couldn't.

I don't really see any limits on these models.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#637
post #490

This is insanely expensive to run though. Looks like it cost around $1 million of compute to get that result. Doesn't seem like such a massive breakthrough when they are throwing so much compute at it, particularly as this is test time compute, it just isn't practical at all, you are not getting this level with a ChatGPT subscription, even the new $200 a month option.

Sure but... this is the technology at the most expensive it will ever be. I'm impressed that o3 was able to achieve such high performance at all, and am not too pessimistic about costs decreasing over time.

We've seen 10-100x cost decrease per year since GPT-3 came out for the same capabilities.

So... Next year this tech will most likely be quite a bit cheaper.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#638

Efficiency is now key. ~=$3400 per single task to meet human performance on this benchmark is a lot. Also it shows the bullets as "ARC-AGI-TUNED", which makes me think they did some undisclosed amount of fine-tuning (eg. via the API they showed off last week), so even more compute went into this task. We can compare this roughly to a human doing ARC-AGI puzzles, where a human will take (high variance in my subjective…

some other imporant quotes: "Average human off the street: 70-80%. STEM college grad: >95%. Panel of 10 random humans: 99-100%" -@fchollet on X So, considering that the $3400/task system isn't able to compete with STEM college grad yet, we still have some room (but it is shrinking, i expect even more compute will be thrown and we'll see these barriers broken in coming years) Also, some other back of envelope calculat…

Don’t forget humans which is real GI paired with increasing capable AI can create a feed back loop to accelerate new advances.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#639
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

Think of AI as an excavator. You know, those machines that dig holes. 70 years ago, those holes would have been dug by 50 men with shovels. Now it's one guy in an excavator. But we don't have mass unemployment. The excavator just creates more work for bricklayers, carpenters etc.

If AI lives up to hype, you could be the excavator driver. Or, the AI will create a ton of upstream and downstream work. There will be no mass unemployment.

Post reply on HN