Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

github.com

Post reply on HN
Print view View original

81–90 of 158 posts

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#81

Post by zozbot234 »

kkzz99 wrote: post #79
zozbot234 wrote: post #70

How does this benchmark against Reflection, which was fine-tuned to do the same thing-- provide a detailed Chain of Thought with self-corrections, then write out a final answer?

Pretty sure Reflection-70B was a complete scam. They did the ole bait and switch. The model that they uploaded was completely under-performing compared to their own benchmarks and the "secret API" was just a GPT-4 & Claude wrapper.

I'm aware of the issue with their purported benchmarks, in fact some testing had Reflection 70B performing a bit worse than plain Llama-3.1 70B. Does G1 do any better?

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#82

Post by killerstorm »

narrator wrote: post #71
thorum wrote: post #32

o1’s innovation is not Chain-of-Thought. It’s teaching the model to do CoT well (from massive amounts of human feedback) instead of just pretending to. You’ll never get o1 performance just from prompt engineering.

If the training data for these LLMs is from humanity in general, and it is trying to imitate humanity, wouldn't its IQ tend to be the average of all of humanity? Perhaps the only people who talk about STEM topics are people of higher IQ generally, including a lot of poor students asking homework questions. Thus, the way to get to higher IQ output is to critique the lower IQ answers, which may be more numerous by reje…

No.

Does Midjourney output look like an average human drawing?

Obviously, OpenAI knows how to train a classifier...

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#83

Post by [deleted] »

sebzim4500 wrote: post #2

>In all-caps to improve prompt compliance by emphesizing the importance of the instruction This kind of thing is still so funny to me. I wonder if the first guy who gets AGI to work will do it by realizing that he can improve LLM reliability over some threshold by telling it in all caps that his pet's life depends on the answer.

[deleted]

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#84

Post by Oras »

thorum wrote: post #32

o1’s innovation is not Chain-of-Thought. It’s teaching the model to do CoT well (from massive amounts of human feedback) instead of just pretending to. You’ll never get o1 performance just from prompt engineering.

Well, with Tree Of Thought (ToT) and fine-tuned models, I'm sure you can achieve the same performance with margin to improve as you identify the bottlenecks.

I'm not convinced OpenAI is using one model. Look at the thinking process (UI), which takes time, and then suddenly, you have the output streamed out at high speed.

But even so, people are after results, not really the underlying technology. There is no difference of doing it with one model vs multiple models.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#85

Post by bratwurst3000 »

astrange wrote: post #63
nell wrote: post #59

Earlier quoted context omitted.

> I wouldn't call o1 a "system". It's a model, but unlike previous models, it's trained to generate a very long chain of thought before returning a final answer https://x.com/polynoamial/status/1834641202215297487

That answer seems to conflict with "in the future we'd like to give users more control over the thinking time". I've gotten mini to think harder by asking it to, but it didn't make a better answer. Though now I've run out of usage limits for both of them so can't try any more…

i am telling claude to give me not the obvious answer. that put thinking time up and the quality of answers is better. hope it helps.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#86

Post by Yiin »

zozbot234 wrote: post #81
kkzz99 wrote: post #79

Earlier quoted context omitted.

Pretty sure Reflection-70B was a complete scam. They did the ole bait and switch. The model that they uploaded was completely under-performing compared to their own benchmarks and the "secret API" was just a GPT-4 & Claude wrapper.

I'm aware of the issue with their purported benchmarks, in fact some testing had Reflection 70B performing a bit worse than plain Llama-3.1 70B. Does G1 do any better?

g1 is not a model, it's a prompt, so not sure what you would be comparing. Claude vs Claude w/ g1 promp?

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#87

Post by regularfry »

anshumankmr wrote: post #57
COAGULOPATH wrote: post #54

Earlier quoted context omitted.

Came here hoping to find this. You will not unlock "o1-like" reasoning by making a model think step by step. This is an old trick that people were using on GPT3 in 2020. If it were that simple, it wouldn't have taken OpenAI so long to release it. Additionally, some of the prompt seems counterproductive: >Be aware of your limitations as an llm and what you can and cannot do. The LLM doesn't have a good idea of its lim…

>The LLM doesn't have a good idea of its limitations (any more than humans do). I expect this will create false refusals, as the model becomes overcautious. Can it not be trained to do so? From my anecdotal observations, the knowledge cutoff is one thing that LLMs are really well trained to know about. Those are limitations that LLMs are currently well trained to handle. Why can it not be trained to know that it is q…

Apart from anything else there will be a lot of text about the nature of LLMs and their inherent limitations in its training set. It might only need to be made salient the fact that it is one in order to produce the required effect.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#88

Post by cabidaher »

cubefox wrote: post #80
segmondy wrote: post #30

This is not even remotely close and very silly. A ChainOfThought in a loop. TreeOfThoughts is a more sophisticated method, see - https://arxiv.org/pdf/2305.10601 The clue we all had with OpenAI for a long time that this was a search through a tree, they hired Noam Brown, and his past work all hinted towards that. Q , is obviously a search on a tree like A . So take something like CoT, build out a tree, search for the…

It's interesting that DeepMind still publishes this stuff. OpenAI doesn't publish anything of that sort anymore. DeepMind is more research/publication focused, but this is a disadvantage in a competitive landscape where OpenAI and Anthropic can just apply the results of your paper without giving anything back to the research community.

Anthropic publishes quite a lot too though.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#89

Post by marricks »

cubefox wrote: post #80
segmondy wrote: post #30

This is not even remotely close and very silly. A ChainOfThought in a loop. TreeOfThoughts is a more sophisticated method, see - https://arxiv.org/pdf/2305.10601 The clue we all had with OpenAI for a long time that this was a search through a tree, they hired Noam Brown, and his past work all hinted towards that. Q , is obviously a search on a tree like A . So take something like CoT, build out a tree, search for the…

It's interesting that DeepMind still publishes this stuff. OpenAI doesn't publish anything of that sort anymore. DeepMind is more research/publication focused, but this is a disadvantage in a competitive landscape where OpenAI and Anthropic can just apply the results of your paper without giving anything back to the research community.

> but this is a disadvantage in a competitive landscape

Or it's a unique advantage because this stuff doesn't happen without good researches who may want:

1) Their name in scientific papers

2) They might actually care about the openess of AI

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#90

Post by cubefox »

marricks wrote: post #89
cubefox wrote: post #80

Earlier quoted context omitted.

It's interesting that DeepMind still publishes this stuff. OpenAI doesn't publish anything of that sort anymore. DeepMind is more research/publication focused, but this is a disadvantage in a competitive landscape where OpenAI and Anthropic can just apply the results of your paper without giving anything back to the research community.

> but this is a disadvantage in a competitive landscape Or it's a unique advantage because this stuff doesn't happen without good researches who may want: 1) Their name in scientific papers 2) They might actually care about the openess of AI

So far it seems to be a disadvantage as DeepMind has fallen behind OpenAI, despite their size, and to some extent even behind Anthropic.
Post reply on HN