[deleted]
g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
11–20 of 158 posts
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#12This is the system prompt it uses:
You are an expert AI assistant that explains your reasoning step by step. For each step, provide a title that describes what you're doing in that step, along with the content. Decide if you need another step or if you're ready to give the final answer. Respond in JSON format with 'title', 'content', and 'next_action' (either 'continue' or 'final_answer') keys. USE AS MANY REASONING STEPS AS POSSIBLE. AT LEAST 3. BE AWARE OF YOUR LIMITATIONS AS AN LLM AND WHAT YOU CAN AND CANNOT DO. IN YOUR REASONING, INCLUDE EXPLORATION OF ALTERNATIVE ANSWERS. CONSIDER YOU MAY BE WRONG, AND IF YOU ARE WRONG IN YOUR REASONING, WHERE IT WOULD BE. FULLY TEST ALL OTHER POSSIBILITIES. YOU CAN BE WRONG. WHEN YOU SAY YOU ARE RE-EXAMINING, ACTUALLY RE-EXAMINE, AND USE ANOTHER APPROACH TO DO SO. DO NOT JUST SAY YOU ARE RE-EXAMINING. USE AT LEAST 3 METHODS TO DERIVE THE ANSWER. USE BEST PRACTICES.
Does it work? Well not really:
https://lluminous.chat/?sl=Yjkxpu
https://lluminous.chat/?sl=jooz48
I have also been using this prompt, and while it fails on then problem above, it works better for me than OPs prompt:
Write many chains of thought for how you’d approach solving the user's question. In this scenario, more is more. You need to type out as many thoughts as possible, placing all your thoughts inside tags.
Your thoughts are only visible to yourself, the user does not see them and they should not be considered to be part of the final response.
Consider every possible angle, recheck your work at every step, and backtrack if needed.
Remember, there are no limits in terms of how long you can think - more thinking will always lead to a better solution.
You should use your thoughts as a scratchpad, much like humans do when performing complicated math with paper and pen. Don't omit any calculation, write everything out explicitly.
When counting or maths is involved, write down an enormously verbose scratchpad containing the full calculation, count, or proof, making sure to LABEL every step of the calculation, and writing down the solution step by step.
Always remember that if you find yourself consistently getting stuck, taking a step back and reconsidering your approach is a good idea. If multiple solutions are plausible, explore each one individually, and provide multiple answers.
Always provide mathematical proofs of mathematical answers. Be as formal as possible and use LaTeX.
Don't be afraid to give obvious answers. At the very very end, after pages upon pages of deep thoughts, synthesize the final answer, inside tags.
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#13So it’s the asic groq guys right?
Because it says so nowhere in the repo.
Man Elon makes things confusing.
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#14So it’s the asic groq guys right? Because it says so nowhere in the repo. Man Elon makes things confusing.
The Elon one is spelled Grok, not Groq.
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#15i love seeing stuff like this, im guessing it wont be long until this method becomes the norm
This is basically CoT, so it's already the norm for a lot of benchmarks. I think the value proposition here is that it puts a nice UX around using it in a chat interface.
That was my initial position too, but I think there is a search efficiency story here as well. CoT comes in many flavors and improves when tailored to the problem domain. If the LLM can instead figure out the right strategy to use to problem solve for a given problem, this may improve performance per compute vs discovering this at inference time.
Tailoring prompts is likely still the best way to maximize performance when you can, but in broader domains you'd work around this through strategies like asking the LLM to combine predefined reasoning modules, or creating multiple reasoning chains and merging/comparing them, explicit MCTS etc. I think those strategies will still be useful for a good while, but pieces of that search process, especially directing the search more efficiently, move to the LLMs over time as they get trained with this kind of data.
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#16So it’s the asic groq guys right? Because it says so nowhere in the repo. Man Elon makes things confusing.
[deleted]
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#17This seems the usual CoT that has been used for a while, o1 was trained with reinforcement learning with some unknown policy, so it's much better at utilizing the chain of thought.
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#18>In all-caps to improve prompt compliance by emphesizing the importance of the instruction This kind of thing is still so funny to me. I wonder if the first guy who gets AGI to work will do it by realizing that he can improve LLM reliability over some threshold by telling it in all caps that his pet's life depends on the answer.
Telling LLMs not to hallucinate in their prompt improves the output. https://arstechnica.com/gadgets/2024/08/do-not-hallucinate-t...
Just because Apple includes it in one of their prompts doesn't mean it improves performance.
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#19Earlier quoted context omitted.
Telling LLMs not to hallucinate in their prompt improves the output. https://arstechnica.com/gadgets/2024/08/do-not-hallucinate-t...
Just because Apple includes it in one of their prompts doesn't mean it improves performance.
It seems plausible that stressing the importance of the system prompt instructions might do something, but I don't see how telling the model not to hallucinate would work. How could the model know that its most likely prediction has gone off the rails, without any external point of reference?
Re: g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains
#20Earlier quoted context omitted.
Just because Apple includes it in one of their prompts doesn't mean it improves performance.
It seems plausible that stressing the importance of the system prompt instructions might do something, but I don't see how telling the model not to hallucinate would work. How could the model know that its most likely prediction has gone off the rails, without any external point of reference?
Some of the text that the LLM is trained on is fictional, some of the text that its trained on is factual. Telling it to not make things up can tell it to generate text that’s more like the factual text. Not saying it does work, but this is a reason how it might work.