Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

github.com

Post reply on HN
Print view View original

41–50 of 158 posts

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#41

Post by arresin »

ttul wrote: post #25
a-dub wrote: post #10

so is this o1 thing just cot (like has been around for a few years) but baked into the training transcripts, rlhf and inference pipeline?

Pasting from my Perplexity page on the topic: The core innovation [1] of o1 lies in its ability to generate and refine internal chains of thought before producing a final output [2]. Unlike traditional LLMs that primarily focus on next-token prediction, o1 learns to: 1. Recognize and correct mistakes 2. Break down complex steps into simpler ones 3. Try alternative approaches when initial strategies fail This process…

That answers nothing the commenter asked.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#42

Post by zaptrem »

segmondy wrote: post #30

This is not even remotely close and very silly. A ChainOfThought in a loop. TreeOfThoughts is a more sophisticated method, see - https://arxiv.org/pdf/2305.10601 The clue we all had with OpenAI for a long time that this was a search through a tree, they hired Noam Brown, and his past work all hinted towards that. Q , is obviously a search on a tree like A . So take something like CoT, build out a tree, search for the…

Where in their blog post (which seemingly had complete examples of the model’s chain of thought) did they suggest they were using search or tree of thoughts?

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#43

Post by hadeson »

arthurcolle wrote: post #36
Jianghong94 wrote: post #26

Earlier quoted context omitted.

yeah or a lot of people can just fake progress by attaching whatever viral tag onto their glue code. I mean to start with, unless you do a bit of fine-tuning + rlhf there's no way to do it o1-like.

no its a lot more than RLHF, I think they figured out a way to have the LLM actually actively plot out scenario trajectories via context window manipulation and then use some kind of adhoc reward shaping mechanism to get it to select the best path based on the user's profile in a way that gets the most likely to be "liked" scenario (context window state change up to some N number of tokens (seems like they've been lo…

I like the Tree of Thoughts theory that treat each chain of thoughts 'branch' as a possible hypothesis. They might trained a search system that quickly explore some of these branches and by some metric choose the most likely to be the right one at the moment to answer.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#44

Post by mrfinn »

tkz1312 wrote: post #40
potatoman22 wrote: post #18

Earlier quoted context omitted.

Just because Apple includes it in one of their prompts doesn't mean it improves performance.

I’ve had pretty good experience with it personally. It quite often just tells me it doesn’t know or isn’t sure instead of just making something up.

I did something similar and to my surprise effectively made the LLM in my tests admit when they don't know something. Not always but worked sometimes. I don't prompt "don't hallucinate" but "admit when you don't know something". It's a logical thing in the other hand, many prompts just transmit the idea of being "helpful" or "powerful" to the LLMs without any counterweight idea. So the LLM tries to say something "helpful" in any case.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#45

Post by halfjoking »

gmt2027 wrote: post #33
tonetegeatinst wrote: post #28

Groq 2 isn't as open as groq 1 iirc. Still hoping we get at least open weights.

You're thinking of Grok, the model from xAI. This Groq is the inference hardware company with a cloud service.

Groq is more refined - it has a “q” in it because it’s got those fancy LPUs.

Grok rhymes with cock, because Elon wants you to use it with your cock out.

That’s how I remember the difference.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#46

Post by Joeri »

zaptrem wrote: post #42
segmondy wrote: post #30

This is not even remotely close and very silly. A ChainOfThought in a loop. TreeOfThoughts is a more sophisticated method, see - https://arxiv.org/pdf/2305.10601 The clue we all had with OpenAI for a long time that this was a search through a tree, they hired Noam Brown, and his past work all hinted towards that. Q , is obviously a search on a tree like A . So take something like CoT, build out a tree, search for the…

Where in their blog post (which seemingly had complete examples of the model’s chain of thought) did they suggest they were using search or tree of thoughts?

Just a guess:

The chain of thought would be the final path through the tree. Interactively showing the thought tokens would give the game away, which is why they don’t show that.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#47

Post by arthurcolle »

hadeson wrote: post #43
arthurcolle wrote: post #36

Earlier quoted context omitted.

no its a lot more than RLHF, I think they figured out a way to have the LLM actually actively plot out scenario trajectories via context window manipulation and then use some kind of adhoc reward shaping mechanism to get it to select the best path based on the user's profile in a way that gets the most likely to be "liked" scenario (context window state change up to some N number of tokens (seems like they've been lo…

I like the Tree of Thoughts theory that treat each chain of thoughts 'branch' as a possible hypothesis. They might trained a search system that quickly explore some of these branches and by some metric choose the most likely to be the right one at the moment to answer.

yeah exactly, MCTS

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#49

Post by pants2 »

sebzim4500 wrote: post #2

>In all-caps to improve prompt compliance by emphesizing the importance of the instruction This kind of thing is still so funny to me. I wonder if the first guy who gets AGI to work will do it by realizing that he can improve LLM reliability over some threshold by telling it in all caps that his pet's life depends on the answer.

Indeed, in the early days of Bard, the only way to get it to output only JSON was to threaten a human life[1].

1. https://x.com/goodside/status/1657396491676164096

Post reply on HN