Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

arxiv.org

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#11

Post by fallingfrog »

pfdietz wrote: post #3
plastic-enjoyer wrote: post #2

> scaling to 1T parameters significantly enhances sample efficiency and performance ceilings; Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence. The human brain has a few billion neurons and uses as much power as a light bulb.

The human brain is estimated to have approaching 10^11 neurons (most of them in the cerebellum). However, a neuron is much more than a single parameter. The brain is estimated to have from 10^14 to 5x10^14 synapses.

True although a lot of those neurons and synapses are in the cerebellum, responsible for motor coordination and or in the visual cortex and so forth. Only a portion are in the language and reasoning areas. LLM's are comparable to human scale now, i think, and if trends continue will swiftly pass us by in the future.

If I had a magic button I would not only pause AI development but set it back 10 years. Sadly I have no influence on events and those who do, don't care about the future of humankind or actively wish us dead.

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#12

Post by retinaros »

plastic-enjoyer wrote: post #2

> scaling to 1T parameters significantly enhances sample efficiency and performance ceilings; Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence. The human brain has a few billion neurons and uses as much power as a light bulb.

everything in LLM world is that right now.

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#13

Post by mapontosevenths »

plastic-enjoyer wrote: post #2

> scaling to 1T parameters significantly enhances sample efficiency and performance ceilings; Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence. The human brain has a few billion neurons and uses as much power as a light bulb.

This is the worst they will ever be.

In 1956 a 5mb hard drive shipped on a large truck and took a team of men to unload. It consumed huge amounts of power, and cost about $3,200/month to run. In today's dollars that would be about $160,000 per month.

Aren't you glad we didnt just give up because it was kind of expensive?

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#14

Post by Alpha3031 »

fallingfrog wrote: post #11
pfdietz wrote: post #3

Earlier quoted context omitted.

The human brain is estimated to have approaching 10^11 neurons (most of them in the cerebellum). However, a neuron is much more than a single parameter. The brain is estimated to have from 10^14 to 5x10^14 synapses.

True although a lot of those neurons and synapses are in the cerebellum, responsible for motor coordination and or in the visual cortex and so forth. Only a portion are in the language and reasoning areas. LLM's are comparable to human scale now, i think, and if trends continue will swiftly pass us by in the future. If I had a magic button I would not only pause AI development but set it back 10 years. Sadly I have n…

More recent research suggests the cerebellum does actually have a significant role in cognition. See for example https://pmc.ncbi.nlm.nih.gov/articles/PMC4089997/ and https://www.sciencedirect.com/science/article/pii/S138904172....

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#15

Post by janalsncm »

I really feel out of my depth because 2 out of the 3 methods here seem like they shouldn’t work?

> To evaluate comprehensibility quantitatively, we employ an LLM-as-a-Judge framework

This isn’t the worst idea, but it’s still a bit incestuous. Adding an LLM judge to check for hallucinations creates two new kinds of problems: false positives, where your judge hallucinates an incorrect fact, and false negatives, where the judge lets a hallucination slip by.

> We measure reproducibility through knowledge distillation. By fine-tuning a weaker model on the generated CoT traces, we use the downstream performance gain of the student as a proxy.

And my problem here, as a member of the GPU proletariat, is that this just seems incredibly inefficient. In other words, you’re going to generate a bunch of rollouts from your model then wait for the student to train? I guess if you have the compute to train a trillion params then maybe you don’t care.

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#16

Post by arcanemachiner »

janalsncm wrote: post #15

I really feel out of my depth because 2 out of the 3 methods here seem like they shouldn’t work? > To evaluate comprehensibility quantitatively, we employ an LLM-as-a-Judge framework This isn’t the worst idea, but it’s still a bit incestuous. Adding an LLM judge to check for hallucinations creates two new kinds of problems: false positives, where your judge hallucinates an incorrect fact, and false negatives, where t…

I have my LLM agents fact check each other as a matter of course. They each regularly find things that the other missed. They are typically the same model (Opus).

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#17

Post by hansvm »

janalsncm wrote: post #15

I really feel out of my depth because 2 out of the 3 methods here seem like they shouldn’t work? > To evaluate comprehensibility quantitatively, we employ an LLM-as-a-Judge framework This isn’t the worst idea, but it’s still a bit incestuous. Adding an LLM judge to check for hallucinations creates two new kinds of problems: false positives, where your judge hallucinates an incorrect fact, and false negatives, where t…

> LLM-as-a-Judge

Empirically, many problems look like they're easier to check than they are to solve. This seems like a reasonable way to bootstrap a little extra performance, with prior art in well-known DeepMind experiments. It's unclear if it works recursively (I imagine not), but the core idea is solid.

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#18

Post by plastic-enjoyer »

mapontosevenths wrote: post #13
plastic-enjoyer wrote: post #2

> scaling to 1T parameters significantly enhances sample efficiency and performance ceilings; Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence. The human brain has a few billion neurons and uses as much power as a light bulb.

This is the worst they will ever be. In 1956 a 5mb hard drive shipped on a large truck and took a team of men to unload. It consumed huge amounts of power, and cost about $3,200/month to run. In today's dollars that would be about $160,000 per month. Aren't you glad we didnt just give up because it was kind of expensive?

Nobody's saying give up. I'm saying if your solution needs a trillion parameters and a power plant, and biology does it with a few billion neurons and a sandwich, that you're maybe on the wrong track. This is not an engineering gap.

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#19

Post by Zababa »

plastic-enjoyer wrote: post #18
mapontosevenths wrote: post #13

Earlier quoted context omitted.

This is the worst they will ever be. In 1956 a 5mb hard drive shipped on a large truck and took a team of men to unload. It consumed huge amounts of power, and cost about $3,200/month to run. In today's dollars that would be about $160,000 per month. Aren't you glad we didnt just give up because it was kind of expensive?

Nobody's saying give up. I'm saying if your solution needs a trillion parameters and a power plant, and biology does it with a few billion neurons and a sandwich, that you're maybe on the wrong track. This is not an engineering gap.

Considering progress in the rest of computing stuff (RAM, CPUs, storage) is kind of "linear"/exponential it sure looks like it's an engineering gap and we're on the right track. GPT 3 was 175B parameters and is today crushed by models that are 32B parameters, that's a lot of progress in 6 years.

Re: Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

#20

Post by dwb »

Eridrus wrote: post #7
plastic-enjoyer wrote: post #2

> scaling to 1T parameters significantly enhances sample efficiency and performance ceilings; Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence. The human brain has a few billion neurons and uses as much power as a light bulb.

Do you find cars similarly hilarious?

I mean yes, cars are absurdly inefficient for a lot of trips. America’s addiction to cars is kind of funny.
Post reply on HN