Live data from Hacker News

OpenAI O3 breakthrough high score on ARC-AGI-PUB

arcprize.org

601–610 of 1001 posts

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#601
post #521

With only a 100x increase in cost, we improved performance by 0.1x and continued plotting this concave-down diminishing-returns type graph! Hurray for logarithmic x-axes! Joking aside, better than ever before at any cost is an achievement, it just doesn't exactly scream "breakthrough" to me.

[flagged]

[flagged]

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#602
post #389

One thing I have not seen commented on is that ARC-AGI is a visual benchmark but LLMs are primarily text. For instance when I see one of the ARC-AGI puzzles, I have a visual representation in my brain and apply some sort of visual reasoning solve it. I can "see" in my mind's eye the solution to the puzzle. If I didn't have that capability, I don't think I could reason through words how to go about solving it - it wou…

Yeah this part is what makes the high performance even more surprising to me. The fact that LLMs are able to do so well on visual tasks (also seen with their ability to draw an image purely using textual output https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ ) implies that not only do they actually have some "world model" but that this is in spite of the disadvantage given by having to fit a round peg in…

A file is a stream of symbols encoded by bits according to some format. It’s pretty much 1D. It would be susprising that LLM couldn’t extract information from a file or a data stream.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#603
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

You have so much time to figure things out. The average person in this thread is probably 1.5-2x your age. I wouldn’t stress too much. AI is an amazing tool. Just use it to make hay while the sun shines, and if it puts you out of work and automates away all other alternatives, then you’ll be witnessing the greatest economic shift in human history. Productivity will become easier than ever, before it becomes automatic…

> if it puts you out of work and automates away all other alternatives, then you’ll be witnessing the greatest economic shift in human history

This is my view but with a less positive spin: you are not going to be the only person whose livelihood will be destroyed. It's going to be bad for a lot of people.

So at least you'll have a lot of company.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#604
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

As long as your chosen profession isn't completing AI benchmarks for money, you should be okay.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#605
post #42

Human performance is 85% [1]. o3 high gets 87.5%. This means we have an algorithm to get to human level performance on this task. If you think this task is an eval of general reasoning ability, we have an algorithm for that now. There's a lot of work ahead to generalize o3 performance to all domains. I think this explains why many researchers feel AGI is within reach, now that we have an algorithm that works. Congrat…

I’ll believe it when the AI can earn money on its own. I obviously don’t mean someone paying a subscription to use the AI I mean, letting the AI lose on the Internet with only the goal of making money and putting it into a bank account.

Do trading bots count?

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#606
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

This is me as well. Either: 1) Just give up computing entirely, the field I've been dreaming about since childhood. Perhaps if I immiserate myself with a dry regulated engineering field or trade I would perhaps survive to recursive self-improvement, but if anything the length it takes to pivot (I am a Junior in College that has already done probably 3/4th of my CS credits) means I probably couldn't get any foothold u…

Dude chill! Eight years ago, I remember driving to some relatives for Thanksgiving and thinking that self-driving cars were just around the corner and how it made no sense for people to learn how to drive semis. Here we are eight years later and self-driving semis aren't a thing--yet. They will be some day, but we aren't there yet.

If you want to work in computing, then make it happen! Use the tools available and make great stuff. Your computing experience will be different from when I graduated from college 25 years ago, but my experience with computers was far different from my Dad's. Things change. Automation changes jobs. So far, it's been pretty good.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#607

Let me go against some skeptics and explain why I think full o3 is pretty much AGI or at least embodies most essential aspects of AGI. What has been lacking so far in frontier LLMs is the ability to reliably deal with the right level of abstraction for a given problem. Reasoning is useful but often comes out lacking if one cannot reason at the right level of abstraction. (Note that many humans can't either when they…

Ask o3 is P=NP?

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#608
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

As engineers, we solve problems. Picking a problem domain close to your heart that intersects with your skills will likely be valued - and valuable. Engage the work, aim to understand and solve the human problems for those around you, and the way forward becomes clearer. Human problems (food, health, safety) are generally constant while tools may change. Learn and use whatever tools to help you, be it scientific principles, hammers or LLMs. For me, doing so and living within my means has been intrinsically satisfying. Not terribly successful materially but has been a good life so far. Good luck.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#609
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

buy bitcoin. when the last job has been automated away, millions of AIs globally will do commerce with each other and they will use bitcoin to pay each other. as long as the human race (including AIs) produces new goods and services, the purchasing power of bitcoin will go up, indefinitely. even more so once we unlock new industries in space (settlements on the Moon and Mars, asteroid mining etc). The only thing that…

This must be a joke since you must know how many people control the majority of bitcoin.

Re: OpenAI O3 breakthrough high score on ARC-AGI-PUB

#610
post #537

I just graduated college, and this was a major blow. I studied Mechanical Engineering and went into Sales Engineering because cause I love technology and people, but articles like this do nothing but make me dread the future. I have no idea what to specialize in, what skills I should master, or where I should be spending my time to build a successful career. Seems like we’re headed toward a world where you automate s…

I think we are pretty far. I am not devaluing the o3 capability but going through actual dataset the definition of "handling novel tasks" is pretty limited. The curse of large context of llms is especially present engineering projects and does not appear it will not end up producing the plans of a bridge, or an industrial process. Sone of tasks with smaller contexts sure can be assisted, but you cant RAG or Agent a full solution for the foreseeable future. O3 adds capability towards agi, but in reality actual infinite context with less intelligence would be more disrupting at a shorter time if one was to choose.
Post reply on HN