Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

github.com

Post reply on HN
Print view View original

61–70 of 158 posts

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#61

Post by euroderf »

morkalork wrote: post #29
sebzim4500 wrote: post #2

>In all-caps to improve prompt compliance by emphesizing the importance of the instruction This kind of thing is still so funny to me. I wonder if the first guy who gets AGI to work will do it by realizing that he can improve LLM reliability over some threshold by telling it in all caps that his pet's life depends on the answer.

We used to be engineers, now we're just monkeys throwing poop at the wall to see what the LLM accepts and obeys.

Opening scene of "2001". Engineer throws poop high in the air, and cue lap dissolve to... a Terminator ?

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#63

Post by astrange »

nell wrote: post #59
pizza wrote: post #53

Earlier quoted context omitted.

Source?

> I wouldn't call o1 a "system". It's a model, but unlike previous models, it's trained to generate a very long chain of thought before returning a final answer https://x.com/polynoamial/status/1834641202215297487

That answer seems to conflict with "in the future we'd like to give users more control over the thinking time".

I've gotten mini to think harder by asking it to, but it didn't make a better answer. Though now I've run out of usage limits for both of them so can't try any more…

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#65

Post by astrange »

wkat4242 wrote: post #22
potatoman22 wrote: post #18

Earlier quoted context omitted.

Just because Apple includes it in one of their prompts doesn't mean it improves performance.

Yeah and some of the other prompts were misspelled and of doubtful use: > In order to make the draft response nicer and complete, a set of question [sic] and its answer are provided," reads one prompt. "Please write a concise and natural reply by modify [sic] the draft response," it continues. This really sounds like a placeholder made up by one engineer until a more qualified team sits down and defines it.

That's not a big problem since it will understand it, and if they already fine tuned the model to work with that prompt it'd get harder to change.

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#66

Post by astrange »

kristianp wrote: post #34
thorum wrote: post #32

o1’s innovation is not Chain-of-Thought. It’s teaching the model to do CoT well (from massive amounts of human feedback) instead of just pretending to. You’ll never get o1 performance just from prompt engineering.

Does o1 need some method to allow it to generate lengthy chains of thought, or does it just do it normally after being trained to do so? If so, I imagine o1 clones could just be fine tunes of llamas initially.

You need an extremely large amount of training data of good CoTs. And there probably is some magic; we know LLMs aren't capable of self reflection and none of the other ones are any good at iterating to a better answer.

Example prompt for that: "give me three sentences that end in 'is'."

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#67

Post by astrange »

4ad wrote: post #12

This is the system prompt it uses: You are an expert AI assistant that explains your reasoning step by step. For each step, provide a title that describes what you're doing in that step, along with the content. Decide if you need another step or if you're ready to give the final answer. Respond in JSON format with 'title', 'content', and 'next_action' (either 'continue' or 'final_answer') keys. USE AS MANY REASONING…

That second prompt is interesting. Not magic though. I tried it with every other model I know and they're still basically unable to do:

* give me three sentences that end in "is"

* tell me the line of Star Spangled Banner that comes before "gave proof through the night"

But they did some good thinking before failing at it…

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#68

Post by PROMISE_237 »

segmondy wrote: post #30

This is not even remotely close and very silly. A ChainOfThought in a loop. TreeOfThoughts is a more sophisticated method, see - https://arxiv.org/pdf/2305.10601 The clue we all had with OpenAI for a long time that this was a search through a tree, they hired Noam Brown, and his past work all hinted towards that. Q , is obviously a search on a tree like A . So take something like CoT, build out a tree, search for the…

[dead]

Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

#69

Post by PROMISE_237 »

worstspotgain wrote: post #6
sebzim4500 wrote: post #2

>In all-caps to improve prompt compliance by emphesizing the importance of the instruction This kind of thing is still so funny to me. I wonder if the first guy who gets AGI to work will do it by realizing that he can improve LLM reliability over some threshold by telling it in all caps that his pet's life depends on the answer.

For extra compliance, use tags, set volume to 11, phasers to 7, and use SchIzOCasE and +E+X+T+R+A+I+M+P+O+R+T+A+N+T+ annotations. That's assuming Unicode is not supported of course.

[dead]
Post reply on HN