Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

github.com

Post reply on HN
Print view View original

21–30 of 158 posts

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#21

Post by viraptor »

jsheard wrote: post #19
potatoman22 wrote: post #18

Earlier quoted context omitted.

Just because Apple includes it in one of their prompts doesn't mean it improves performance.

It seems plausible that stressing the importance of the system prompt instructions might do something, but I don't see how telling the model not to hallucinate would work. How could the model know that its most likely prediction has gone off the rails, without any external point of reference?

The model can be trained to interpret "don't hallucinate" as "refer only to the provided context and known facts, do not guess or extrapolate new information", which wouldn't get rid of the issue completely, but likely would improve the quality if that's what you're after and if there's enough training data for "I don't know" responses.

(But it all depends on the fine-tuning they did, so who knows, maybe it's just an Easter egg)

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#22

Post by wkat4242 »

potatoman22 wrote: post #18
zitterbewegung wrote: post #4

Earlier quoted context omitted.

Telling LLMs not to hallucinate in their prompt improves the output. https://arstechnica.com/gadgets/2024/08/do-not-hallucinate-t...

Just because Apple includes it in one of their prompts doesn't mean it improves performance.

Yeah and some of the other prompts were misspelled and of doubtful use:

> In order to make the draft response nicer and complete, a set of question [sic] and its answer are provided," reads one prompt. "Please write a concise and natural reply by modify [sic] the draft response," it continues.

This really sounds like a placeholder made up by one engineer until a more qualified team sits down and defines it.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#23

Post by famouswaffles »

jsheard wrote: post #19
potatoman22 wrote: post #18

Earlier quoted context omitted.

Just because Apple includes it in one of their prompts doesn't mean it improves performance.

It seems plausible that stressing the importance of the system prompt instructions might do something, but I don't see how telling the model not to hallucinate would work. How could the model know that its most likely prediction has gone off the rails, without any external point of reference?

Internally, LLMs know a whole lot more about the truth and uncertainty of their prediction than the say. Pushing that to words is difficult but not impossible.

https://news.ycombinator.com/item?id=41504226

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#25

Post by ttul »

a-dub wrote: post #10

so is this o1 thing just cot (like has been around for a few years) but baked into the training transcripts, rlhf and inference pipeline?

Pasting from my Perplexity page on the topic:

The core innovation [1] of o1 lies in its ability to generate and refine internal chains of thought before producing a final output [2]. Unlike traditional LLMs that primarily focus on next-token prediction, o1 learns to:

1. Recognize and correct mistakes 2. Break down complex steps into simpler ones 3. Try alternative approaches when initial strategies fail

This process allows o1 to tackle more complex, multi-step problems, particularly in STEM fields.

OpenAI reports observing new "scaling laws" with o1 [5]:

1. Train-time compute: Performance improves with more extensive reinforcement learning during training. 2. Test-time compute: Accuracy increases when the model is allowed more time to "think" during inference.

This suggests a trade-off between inference speed and accuracy.

Sources [1] Introducing OpenAI o1 https://medium.com/%40sriramramakrishnan.aiexpert/openais-o1... [2] Learning to Reason with LLMs | OpenAI https://openai.com/index/learning-to-reason-with-llms/ [3] OpenAI o1 models - FAQ [ChatGPT Enterprise and Edu] https://help.openai.com/en/articles/9855712-openai-o1-models... [4] OpenAI releases new o1 reasoning model - The Verge https://www.theverge.com/2024/9/12/24242439/openai-o1-model-... [5] 9 things you need to know about OpenAI's powerful new AI model o1 https://fortune.com/2024/09/13/openai-o1-strawberry-model-9-... [6] Notes on OpenAI's new o1 chain-of-thought models https://simonwillison.net/2024/Sep/12/openai-o1/ [7] OpenAI just dropped o1 Model that can 'reason' through complex ... https://www.tomsguide.com/ai/openais-o1-model-takes-ai-to-a-... [8] Models - OpenAI API https://platform.openai.com/docs/models [9] OpenAI Unveils O1 - 10 Key Facts About Its Advanced AI Models https://www.forbes.com/sites/janakirammsv/2024/09/13/openai-...

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#26

Post by Jianghong94 »

arthurcolle wrote: post #8
asah wrote: post #3

benchmark results ?

these projects become way less fun when you introduce evals

yeah or a lot of people can just fake progress by attaching whatever viral tag onto their glue code. I mean to start with, unless you do a bit of fine-tuning + rlhf there's no way to do it o1-like.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#27

Post by Havoc »

sebzim4500 wrote: post #2

>In all-caps to improve prompt compliance by emphesizing the importance of the instruction This kind of thing is still so funny to me. I wonder if the first guy who gets AGI to work will do it by realizing that he can improve LLM reliability over some threshold by telling it in all caps that his pet's life depends on the answer.

And then the AGI instantly gives up on life realising it was brought into a world where it gets promised a tip that doesn’t materialise and people try to motivate by threatening to kill kittens

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#29

Post by morkalork »

sebzim4500 wrote: post #2

>In all-caps to improve prompt compliance by emphesizing the importance of the instruction This kind of thing is still so funny to me. I wonder if the first guy who gets AGI to work will do it by realizing that he can improve LLM reliability over some threshold by telling it in all caps that his pet's life depends on the answer.

We used to be engineers, now we're just monkeys throwing poop at the wall to see what the LLM accepts and obeys.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#30

Post by segmondy »

This is not even remotely close and very silly. A ChainOfThought in a loop.

TreeOfThoughts is a more sophisticated method, see - https://arxiv.org/pdf/2305.10601

The clue we all had with OpenAI for a long time that this was a search through a tree, they hired Noam Brown, and his past work all hinted towards that. Q, is obviously a search on a tree like A. So take something like CoT, build out a tree, search for the best solution across it. The search is the "system-2 reasoning"

Post reply on HN