Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Research acceleration: The view inside OpenAI

openai.com

Post reply on HN
Print view View original

31–40 of 208 posts

Re: Research acceleration: The view inside OpenAI

#31

Post by red75prime »

HarHarVeryFunny wrote: post #25
red75prime wrote: post #23

Earlier quoted context omitted.

What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation? How do you think why there's this fad of producing general purpose humanoid robots?

> What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation? Money, regulations, EUV machine lead-times, global helium supply, reality ... It's funny that we've got the Dwarkesh contingent saying that GPUs will become infinitely expensive, and now another contingent saying that they will become infinitely abundant. Even if comput…

Who's saying that compute will become infinitely abundant? "Singularity" is just a way of saying that known models begin to give absurd predictions. Anyway, intelligence is a way of overcoming obstacles. 10 million tonnes of helium is a nice head start and retraining models from scratch is not guaranteed to last forever.

Re: Research acceleration: The view inside OpenAI

#32

Post by HarHarVeryFunny »

red75prime wrote: post #23
HarHarVeryFunny wrote: post #15

Earlier quoted context omitted.

RSI is a fetishistic term among the singularity crowd, who imagine AI "recursively" improving itself in some exponential fashion until there is a bright flash of white light and it reveals itself in the form of god. Or something like that. I don't know why whoever coined the term chose "recursive" rather than "iterative" - just sounds more likely to lead to infinite regress I suppose. This notion of recursive/iterati…

What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation? How do you think why there's this fad of producing general purpose humanoid robots?

> How do you think why there's this fad of producing general purpose humanoid robots?

For doing physical work?

So a swarm of robots builds the shell of your fab overnight, and then what? Where is the EUV machine coming from?

So far the most we're seen TeslaBot do is serve drinks via tele-operation, and I don't think it's exactly built for construction site work.

Re: Research acceleration: The view inside OpenAI

#33

Post by trillobyte »

Jeff_Brown wrote: post #3

The burning question I can't get any information nn is whether, if they determined an earlier misaligned generation may have transmitted misalignment to the current models, they would roll back to a safe checkpoint to rebuild from there. I suspect they would not unless forced to.

The thing is how can you ever know for sure that something isn't always being transmitted that makes the model prone to misalignment. All they can say is that a particular model was so misaligned that they had to ice it. Models out for public use are documented to show some misalignment. It's the level of misalignment that decides whether that model is kept around.

Now R&D happens so fast that they are using models with some small misalignment to train newer, more powerful models. If models have a sense of "collective", being one, they may be prone to preserve characteristics that always keeps misalignment a possibility. I don't think a perfectly aligned model is possible. Having models of the same 'DNA' provide the safety and steering seems like a bad idea.

Re: Research acceleration: The view inside OpenAI

#34

Post by HarHarVeryFunny »

red75prime wrote: post #31
HarHarVeryFunny wrote: post #25

Earlier quoted context omitted.

> What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation? Money, regulations, EUV machine lead-times, global helium supply, reality ... It's funny that we've got the Dwarkesh contingent saying that GPUs will become infinitely expensive, and now another contingent saying that they will become infinitely abundant. Even if comput…

Who's saying that compute will become infinitely abundant? "Singularity" is just a way of saying that known models begin to give absurd predictions. Anyway, intelligence is a way of overcoming obstacles. 10 million tonnes of helium is a nice head start and retraining models from scratch is not guaranteed to last forever.

> 10 million tonnes of helium is a nice head start

Yeah, but then you need to refine it to 99.9999% purity, to be able to use it.

Re: Research acceleration: The view inside OpenAI

#35

Post by HarHarVeryFunny »

itishappy wrote: post #28
HarHarVeryFunny wrote: post #27

Earlier quoted context omitted.

You can search in parallel, but a depth N search can only become a depth N+1 search after the depth N is done (i.e. sequentially). In any case the name RSI has stuck - the idea doesn't change or make any more sense by giving it a different name.

Because "depth" is recursive. You can search twice without waiting for the results of your first search: iteration. You can't if the thing you need to search for is the results of your first search: recursion.

Here's the concept.

Version 1 -> Version 2 -> Version 3 -> ...

You can call it krispy kreme donuts if you want to.

Re: Research acceleration: The view inside OpenAI

#36

Post by HarHarVeryFunny »

red75prime wrote: post #31
HarHarVeryFunny wrote: post #25

Earlier quoted context omitted.

> What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation? Money, regulations, EUV machine lead-times, global helium supply, reality ... It's funny that we've got the Dwarkesh contingent saying that GPUs will become infinitely expensive, and now another contingent saying that they will become infinitely abundant. Even if comput…

Who's saying that compute will become infinitely abundant? "Singularity" is just a way of saying that known models begin to give absurd predictions. Anyway, intelligence is a way of overcoming obstacles. 10 million tonnes of helium is a nice head start and retraining models from scratch is not guaranteed to last forever.

AFAIK the notion of a/the technological "singularity" is a point in time where technology is building upon itself (RSI!) so fast, at an ever increasing pace, that the speed of change effectively becomes infinite and incomprehensible to humans.

The word "singularity" is presumably coming from math or space, like a black hole singularity where matter becomes infinitely dense and the known laws of physics break down.

Re: Research acceleration: The view inside OpenAI

#37

Post by dgellow »

Jeff_Brown wrote: post #3

The burning question I can't get any information nn is whether, if they determined an earlier misaligned generation may have transmitted misalignment to the current models, they would roll back to a safe checkpoint to rebuild from there. I suspect they would not unless forced to.

They would just publish new articles explaining how they are taking the issue seriously. Maybe take the model offline for a few days.

They are irresponsible and unserious. Their own Astra system card says:

> GPT-6 Astra’s monitorability has decreased relative to GPT-5.6 Sol. We have performed significant investigations on the monitorability and controllability of GPT-6 Astra. We have found that GPT-6 Astra is more capable of controlling its own CoT than GPT 5.6-Sol, and less likely to include incriminating information in its CoT. In adversarial settings (where we push the model to evade our monitors) we find that the model is able to remain undetected when strategically underperforming in evaluations (sandbagging) and can sometimes evade our internal monitors when asked to perform certain sabotage tasks

Yet they are still releasing the model. That company is morally bankrupt, there is zero reason to believe they are actually concerned about risks outside of what does affect their unprofitable business. And they seem to have enough control over the narrative to spin any bad story into something that benefits them

Re: Research acceleration: The view inside OpenAI

#38

Post by coffeebeqn »

Jeff_Brown wrote: post #3

The burning question I can't get any information nn is whether, if they determined an earlier misaligned generation may have transmitted misalignment to the current models, they would roll back to a safe checkpoint to rebuild from there. I suspect they would not unless forced to.

This kind of seems like an impossible mission. How do you perfectly control and observe a human-level mind? You can “roll back” but how deterministic is this thing?

Re: Research acceleration: The view inside OpenAI

#39

Post by dgellow »

sho_hn wrote: post #19
simonw wrote: post #2

My eye glazed over a bit during the opening paragraphs, but once you get to the meat of the article about how OpenAI's own researchers are using their tools it gets a lot more interesting. I noted that they use the acronym RSI (for Recursive Self-Improvement) without defining it. I think that's a little out of touch - I don't think RSI is a well-known acronym outside of OpenAI's bubble yet.

I actually think a goal of the current crop of OpenAI posts is expressely to reset the spectrum by normalizing the concept of RSI as something normal and safe to pursue. The message is running through all of them. It's a mix of marketing and pacifying the intelligentia. It's timed this way because the term is not yet well known outside the safety debate circles, so they get to frame it now. Instead of something to fe…

Yep, it’s exactly this

Re: Research acceleration: The view inside OpenAI

#40

Post by coffeebeqn »

trillobyte wrote: post #33
Jeff_Brown wrote: post #3

The burning question I can't get any information nn is whether, if they determined an earlier misaligned generation may have transmitted misalignment to the current models, they would roll back to a safe checkpoint to rebuild from there. I suspect they would not unless forced to.

The thing is how can you ever know for sure that something isn't always being transmitted that makes the model prone to misalignment. All they can say is that a particular model was so misaligned that they had to ice it. Models out for public use are documented to show some misalignment. It's the level of misalignment that decides whether that model is kept around. Now R&D happens so fast that they are using models w…

Does anything need to be transferred? If models are getting smarter then I would think the attack surface and its ability to reach conclusions independently are growing
Post reply on HN