Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

github.com

Post reply on HN
Print view View original

131–140 of 158 posts

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#131

Post by fudged71 »

anshumankmr wrote: post #57
COAGULOPATH wrote: post #54

Earlier quoted context omitted.

Came here hoping to find this. You will not unlock "o1-like" reasoning by making a model think step by step. This is an old trick that people were using on GPT3 in 2020. If it were that simple, it wouldn't have taken OpenAI so long to release it. Additionally, some of the prompt seems counterproductive: >Be aware of your limitations as an llm and what you can and cannot do. The LLM doesn't have a good idea of its lim…

>The LLM doesn't have a good idea of its limitations (any more than humans do). I expect this will create false refusals, as the model becomes overcautious. Can it not be trained to do so? From my anecdotal observations, the knowledge cutoff is one thing that LLMs are really well trained to know about. Those are limitations that LLMs are currently well trained to handle. Why can it not be trained to know that it is q…

It's a chicken and egg situation. You don't know a model's capabilities until it is trained. When you then change the training with that learning, it will have modified capabilities.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#132

Post by orbital-decay »

whimsicalism wrote: post #124
quantadev wrote: post #110

Earlier quoted context omitted.

OpenAI is of course going to claim what they've done is very special and hard to replicate. They're a for-profit company and they want to harm the competition any way they can. If they were just doing prompt engineering and multiple inferences they'd definitely want to keep that a competitive secret and send all the open source devs off in random directions, or keep them guessing, rather than telling them which way t…

nobody has shown CoT scaling like this except deepmind, it is very obviously a result of their alignment pipeline not just prompting.

Scaling like what? Are there any comparisons with and without CoT, or with other models with their CoT? As far as I'm aware, their CoT part is secret. I'm sure the finetuning does some lifting, but I'm also sure the difference in a fair comparison won't be remotely as significant as it's being hyped currently.

This is still clearly CoT, with all its limitations and caveats as expected. That's an improvement, sure, but definitely not a qualitative leap like OAI is trying to present it. (in a really shady manner)

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#133

Post by quantadev »

whimsicalism wrote: post #120
quantadev wrote: post #108

Earlier quoted context omitted.

I don't think you can claim you know what's happening internally when OpenAI processes a request. They are a competitive company and will lie for competitive reasons. Most people think Q-Star is doing multiple inferences to accomplish a single task, and that's what all the evidence suggests. Whatever Sam Altman says means absolutely nothing, but I don't think he's claimed they use only a single inference either.

what is “all the evidence”? please share

I recommend getting on Twitter to follow closely the leading individuals in the field of AI, and also watch the leading Youtube channels dedicated to AI research.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#134

Post by quantadev »

whimsicalism wrote: post #124
quantadev wrote: post #110

Earlier quoted context omitted.

OpenAI is of course going to claim what they've done is very special and hard to replicate. They're a for-profit company and they want to harm the competition any way they can. If they were just doing prompt engineering and multiple inferences they'd definitely want to keep that a competitive secret and send all the open source devs off in random directions, or keep them guessing, rather than telling them which way t…

nobody has shown CoT scaling like this except deepmind, it is very obviously a result of their alignment pipeline not just prompting.

For example, a team of GPT3.5 agents can outperform GPT4o. A single inference is essentially just kind of a chain reaction where once you have a set of tokens generated, as it's building an answer, it's looking for next tokens only, and can't revise or rethink. CoT will always outperform the single inference approach.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#135

Post by michaelt »

zozbot234 wrote: post #113
michaelt wrote: post #104

Earlier quoted context omitted.

> They feel behind because they didn't have the smart guy with a new idea a few years back, and HE decided to work at a place which started as open. The "Attention Is All You Need" guys all worked at Google. Google is where they are despite having the smart guys with a new idea a few years back. Of course, IMHO it wouldn't have have helped Google if they'd kept the transformer architecture secret. They'd have fumbled…

Didn't Google have the LaMDA model pretty early, which was even described as "sentient" at some point? That doesn't look "fumbled" to me.

What Google did was sit on their ass, not deigning to release anything. In the meantime, OpenAI became a $150 billion company. And Anthropic came out with Claude, and Facebook with Llama, and Mistral with their models.

Only then did Google realise there might be something to this LLM stuff - so they responded with Bard, a product so poorly received they later had to completely rebrand it. Looks like they didn't have a "sentient" model up their sleeve after all. Then the updated, rebranded model had a bunch of image generation embarrassments of its own.

Admittedly, they have recovered somewhat since then; they're second on some performance leaderboards, which is respectable.

But there was a real tortoise-and-hare situation where they thought they were so far ahead they had time for a nap, until they got overtaken. Any lead they had from inventing transformers and being the only people with TPUs has been squandered.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#136

Post by quantadev »

parineum wrote: post #129
quantadev wrote: post #110

Earlier quoted context omitted.

OpenAI is of course going to claim what they've done is very special and hard to replicate. They're a for-profit company and they want to harm the competition any way they can. If they were just doing prompt engineering and multiple inferences they'd definitely want to keep that a competitive secret and send all the open source devs off in random directions, or keep them guessing, rather than telling them which way t…

> and they want to harm the competition any way they can. That's an incredibly cynical choice of phrasing. Of course they don't want to help the competition, that's what a competition is. The competition isn't helping OpenAI either.

It's not cynical to simply remind everyone who and what is motivating OpenAI (i.e. ClosedAI) at this point. They're no longer about helping the "AI community". They're about holding back from the community. Like you said: "That's what competition is."

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#137

Post by whimsicalism »

ricardobeat wrote: post #127
whimsicalism wrote: post #119

Earlier quoted context omitted.

as someone who works in this field, this comment is obviously uninformed even about old public research trends

Care to elaborate? Your comment would be a lot more useful if it included a little why . Otherwise it’s just teasing readers and at the same time smearing the author without anything to back it up.

reinforcement learning with ppo doesn’t involve mcts and has been the bread and butter of aligning LLMs since 2020. nothing about saying they use rl implies mcts

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#138

Post by whimsicalism »

quantadev wrote: post #133
whimsicalism wrote: post #120

Earlier quoted context omitted.

what is “all the evidence”? please share

I recommend getting on Twitter to follow closely the leading individuals in the field of AI, and also watch the leading Youtube channels dedicated to AI research.

can you link to one speculating about multiple inferences for their CoT? i am curious

e: answer to my own question https://x.com/_xjdr/status/1835352391648158189

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#139

Post by quantadev »

whimsicalism wrote: post #138
quantadev wrote: post #133

Earlier quoted context omitted.

I recommend getting on Twitter to follow closely the leading individuals in the field of AI, and also watch the leading Youtube channels dedicated to AI research.

can you link to one speculating about multiple inferences for their CoT? i am curious e: answer to my own question https://x.com/_xjdr/status/1835352391648158189

So far it's been unanimous. Everyone I've heard talk about it believes Strawberry is mainly just CoT. I'm not saying they didn't fine tune a model too, I'm just saying I agree with most people that clever CoT is where most of the leap in capability seems to have come from.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#140

Post by Meganet »

Hugsun wrote: post #115
Meganet wrote: post #96

Earlier quoted context omitted.

You actually don't know that. A LLM has a huge amount of data ingested. It can create character profiles, audience, personas etc. Why wouldn't it have potentially even learned to 'understand' what 'being aware of your limitations' means? Right now for me 'change of reasoning' feels a little bit of quering the existing meta space through the reasoning process to adjust weights. Basically priming the model. I would als…

It might understand the concept of it having limitations, but it can't AFAIK reliably recognize when it does or doesn't know something, or has encountered a limitation.

Its the same thing as with humans, thats right. It doesn't do Logical reasoning but even the best humans stop at some level.

But if you read all the knowledge of humans, were does your reasoning start? Probably at a very high level of it.

If you look at human brains, we conduct experiments right? As a software developer, we write tests. ChatGPT can already run python code and it can write unit tests.

We do not use proofs when we develop. An AI could actually doing this. But at the end its more of a question who does it better, faster and cheaper eh?

Post reply on HN