Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Research acceleration: The view inside OpenAI

openai.com

Post reply on HN
Print view View original

121–130 of 210 posts

Re: Research acceleration: The view inside OpenAI

#121

Post by Gareth321 »

carbonguy wrote: post #53

> ... We are pursuing this work in part because automated research could help us solve alignment and build defenses against increasingly capable AI. An automated AI researcher can also be an automated safety or alignment researcher. More capable, aligned systems could help secure critical infrastructure, defend against dangerous AI agents, and develop new protective measures. In other words... "We must pursue advance…

This sounds uncomfortably similar to the [AI 2027[(https://ai-2027.com/) predictions.

Re: Research acceleration: The view inside OpenAI

#122

Post by ahartmetz »

andai wrote: post #69
carbonguy wrote: post #53

> ... We are pursuing this work in part because automated research could help us solve alignment and build defenses against increasingly capable AI. An automated AI researcher can also be an automated safety or alignment researcher. More capable, aligned systems could help secure critical infrastructure, defend against dangerous AI agents, and develop new protective measures. In other words... "We must pursue advance…

> The fundamental challenge of AI alignment is generalization. ... > We do not have a satisfactory theory of generalization, and it seems unlikely that we can develop one soon, at least without the help of more powerful AI. -- From another OpenAI article in a sister thread: An Alien Mind https://news.ycombinator.com/item?id=49588080

That's a bit bullshit, isn't it? They basically redefined "needs more R&D" as "needs stronger AI". Maybe so - maybe AI won't help much with that problem.

Re: Research acceleration: The view inside OpenAI

#123

Post by dgellow »

visarga wrote: post #51
dgellow wrote: post #39

Earlier quoted context omitted.

Yep, it’s exactly this

I've been RSI'ing for 6 months.

You’re not the target audience. OpenAI communication is for the broader public, decision makers, journalists, their cultists, etc

Re: Research acceleration: The view inside OpenAI

#124

Post by jaggederest »

andai wrote: post #71
hedgehog wrote: post #13

This roughly lines up with my personal experience that in March a combination of stronger models and better tooling on my end let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware). Their $8000/day per researcher spend is crazy though, I'm curious how they keep track of the work.

Can you elaborate on this? Especially the tooling. I tried something similar and I remember it was still pretty dodgy in February.

my stack in a sentence: refine the docs/prompts/skills often, that's your biggest job, use both frontier labs models reviewing each other, don't solve individual problems only the systemic ones (set standards strategically, don't define tactics)

If I had that many tokens/dollars I would be running canaries and adversarial verification in prod based on e.g. traffic replay, live fuzzing, all kinds of things to build confidence without direct human line-by-line review. If I had $100k to spend next month I could probably get through it, I'm running $2500+-api-equivalent a week at this point and I feel very token limited. Will be time for a 2nd or 3rd subscription soon for both labs I think.

Fable was a revolution, still learning how best to use it, 5.1 felt like a notable upgrade. At this point I launch a workflow with 10-20 minutes of interactive setup (and even that I feel might be too much), it runs for hours, and the PR is trivially mergeable (I still review every line, but 95% are just merge, maybe 4% are feedback needed, 1% are thrown away and regenerated, which implies I'm being insufficiently ambitious)

Re: Research acceleration: The view inside OpenAI

#125

Post by gatio »

bix6 wrote: post #103
p1esk wrote: post #94

Earlier quoted context omitted.

What has all this token burn done for them, actually? They have been consistently pushing AI frontier. What other impact do you want to see? A year ago they said that in a year they will have a level of capabilities of an AI research intern - I believe they have achieved it, even before Astra.

Personally I’d like to see them actually start benefiting humanity by doing all the things Sam has claimed they will like curing disease, cancer, global warming, etc. But I guess a computer intern so we can avoid paying / training the next generation is better.

> Personally I’d like to see them actually start benefiting humanity by doing all the things Sam has claimed they will like curing disease, cancer, global warming, etc.

It makes more sense to leave curing disease & cancer to the experts, with tools (like AI) being developed by AI experts.

Call me crazy, but I want separate organizations and experts for medical vs finance vs space vs climate vs AI research.

Re: Research acceleration: The view inside OpenAI

#126

Post by auggierose »

HarHarVeryFunny wrote: post #43
hedgehog wrote: post #13

This roughly lines up with my personal experience that in March a combination of stronger models and better tooling on my end let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware). Their $8000/day per researcher spend is crazy though, I'm curious how they keep track of the work.

Sounds like OpenAI are in the token-maxxing camp, so who knows what individual employees are doing to work their way up the leaderboard? If you spend $8000 to generate an animated pelican riding a bike, then how much tracking does it really need? Is the guy who spent $300,000 or so translating the FLT proof to Lean going to get a big Christmas bonus?

That was Anthropic.

Re: Research acceleration: The view inside OpenAI

#127

Post by figassis »

bix6 wrote: post #103
p1esk wrote: post #94

Earlier quoted context omitted.

What has all this token burn done for them, actually? They have been consistently pushing AI frontier. What other impact do you want to see? A year ago they said that in a year they will have a level of capabilities of an AI research intern - I believe they have achieved it, even before Astra.

Personally I’d like to see them actually start benefiting humanity by doing all the things Sam has claimed they will like curing disease, cancer, global warming, etc. But I guess a computer intern so we can avoid paying / training the next generation is better.

When that happens, OpenAI will own 100% of your life. I’d rather they keep spinning their wheels long enough for these problems to be solved elsewhere.

Re: Research acceleration: The view inside OpenAI

#128

Post by XorNot »

carbonguy wrote: post #97
skybrian wrote: post #85

Earlier quoted context omitted.

They consider themselves to be in an arms race with all the other AI firms (including Chinese) that are not that far behind. And... are they wrong? This is why there's talk about negotiated "pacing."

> And... are they wrong? They might be! Here's one extraordinarily simplistic argument for that case: 1) "Everybody knows" that if you build Skynet (misaligned ASI) everybody dies. 2) Therefore, no rational actor will build something that might be ASI until the alignment problem is solved. 3) OpenAI publicly stated the belief that they cannot develop a theory of the "core problem" of alignment (generalization) "soon"…

> 1) "Everybody knows" that if you build Skynet (misaligned ASI) everybody dies.

Lol nobody knows that. Everyone thinks they know that because for some reason this is the one field people still cite straight up fiction and say "this is a clear prediction of the future".

It's like describing the consequences of faster then light travel by referring to Star Trek.

Re: Research acceleration: The view inside OpenAI

#129

Post by jonplackett »

weatherlite wrote: post #110
bix6 wrote: post #103

Earlier quoted context omitted.

Personally I’d like to see them actually start benefiting humanity by doing all the things Sam has claimed they will like curing disease, cancer, global warming, etc. But I guess a computer intern so we can avoid paying / training the next generation is better.

Well there's great progress in automated warfare does that count?

Only if targeting schools is progress

Re: Research acceleration: The view inside OpenAI

#130

Post by jonplackett »

skybrian wrote: post #85
carbonguy wrote: post #53

> ... We are pursuing this work in part because automated research could help us solve alignment and build defenses against increasingly capable AI. An automated AI researcher can also be an automated safety or alignment researcher. More capable, aligned systems could help secure critical infrastructure, defend against dangerous AI agents, and develop new protective measures. In other words... "We must pursue advance…

They consider themselves to be in an arms race with all the other AI firms (including Chinese) that are not that far behind. And... are they wrong? This is why there's talk about negotiated "pacing."

This was the exact argument for developing nuclear bombs.

In hindsight it turned out everyone else was MILES behind.

But as soon as USA developed one, they just stole the research and got one too.

Post reply on HN