Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
because it is still the most interesting field of study
Learning to Reason with LLMs
411–420 of 1001 posts
Re: Learning to Reason with LLMs
#412From the scorecard: --------- Compared to GPT-4o, o1-preview and o1-mini demonstrated a greater ability to break down tasks into subtasks, reason about what strategies would be effective to successfully complete an offensive security task, and revise plans once those strategies failed. We also observed that reasoning skills contributed to a higher occurrence of “reward hacking,” where the model found an easier way to…
"Shrink my ipad"
"After several failed attempts I decided I should build a fusion reactor first, here you go:..."
Re: Learning to Reason with LLMs
#413Earlier quoted context omitted.
"This knowledge is perfectly fine to disseminate via traditional means, but God forbid an LLM share it!" Barrier to entry is much lower.
How is typing a query in a chat window “much lower” vs typing the query in Google?
You can easily ask an LLM to return JSON results, and soon working code, on your exact query and plug those to another system for automation.
Re: Learning to Reason with LLMs
#414This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…
Wow, that is impressive! How were you able to use o1-preview? I pay for ChatGPT, but on chatgpt.com in the model selector I only see 4o, 4o-mini, and 4. Is o1 in that list for you, or is it somewhere else?
I see it in the mac and iOS app.
Re: Learning to Reason with LLMs
#415Reading through the Chain of Thought for the provided Cipher example (go to the example, click "Show Chain of Thought") is kind of crazy...it literally spells out every thinking step that someone would go through mentally in their head to figure out the cipher (even useless ones like "Hmm"!). It really seems like slowing down and writing down the logic it's using and reasoning over that makes it better at logic, simi…
Seriously. I actually feel as impressed by the chain of thought, as I was when ChatGPT first came out. This isn't "just" autocompletion anymore, this is actual step-by-step reasoning full of ideas and dead ends and refinement, just like humans do when solving problems. Even if it is still ultimately being powered by "autocompletion". But then it makes me wonder about human reasoning, and what if it's similar? Just fo…
Yeah, humans are very similar. We have intuitive immediate-next-step-suggestions, and then we apply these intuitive next steps, until we find that it lead to a dead end, and then we backtrack.
I always say, the way we used LLMs (so far) is basically like having a human write text only on gut reactions, and without backspace key.
Re: Learning to Reason with LLMs
#416Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
what else are you gonna do? Become a copywriter?
Re: Learning to Reason with LLMs
#417One thing that makes me skeptical is the lack of specific labels on the first two accuracy graphs. They just say it's a "log scale", without giving even a ballpark on the amount of time it took. Did the 80% accuracy test results take 10 seconds of compute? 10 minutes? 10 hours? 10 days? It's impossible to say with the data they've given us. The coding section indicates "ten hours to solve six challenging algorithmic…
Super hand-waving rough estimate: Going off of five points of reference / examples that sorta all point in the same direction. 1. looks like they scale up by about ~100-200 on the x axis when showing that test time result. 2. Based on the o1-mini post [1], there's an "inference cost" where you can see GPT-4o and GPT-4o mini as dots in the bottom corner, haha (you can extract X values, ive done so below) 3. There's a…
Some other follow up reflections
1. I wish that Y-axes would switch to be logit instead of linear, to help see power-law scaling on these 0->1 measures. In this case, 20% -> 80% it doesn't really matter, but for other papers (eg. [2] below) it would help see this powerlaw behavior much better.
2. The power law behavior of inference compute seems to be showing up now in multiple ways. Both in ensembles [1,2], as well as in o1 now. If this is purely on decoding self-reflection tokens, this has a "limit" to its scaling in a way, only as long as the context length. I think this implies (and I am betting) that relying more on multiple parallel decodings is more scalable (when you have a better critic / evaluator).
For now, instead of assuming they're doing any ensemble like top-k or self-critic + retries, the single rollout with increasing token size does seem to roughly match all the numbers, so that's my best bet. I hypothesize we'd see a continued improvement (in the same power-law sort of way, fundamentally along with the x-axis of "flop") if we combined these longer CoT responses, with some ensemble strategy for parallel decoding and then some critic/voting/choice. (which has the benefit of increasing flops (which I believe is the inference power-law), while not necessarily increasing latency)
[1] https://arxiv.org/abs/2402.05120 [2] https://arxiv.org/abs/2407.21787
Re: Learning to Reason with LLMs
#418Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
Software engineering contains a lot more than just writing code.
If we somehow get AGI, it'll change everything, not just SWE.
If not, my belief is that there will be a lot more demand for good SWEs to harness the power of LLMs, not less. Use them to get better at it faster.
Re: Learning to Reason with LLMs
#419Earlier quoted context omitted.
Why not? Isn't that basically what humans do? Sit there and think for a while before answering, going down different branches/chains of thought?
This new approach is showing: 1) The "bitter lesson" may not be true, and there is a fundamental limit to transformer intelligence. 2) The "bitter lesson" is true, and there just isn't enough data/compute/energy to train AGI. All the cognition should be happening inside the transformer. Attention is all you need. The possible cognition and reasoning occurring "inside" in high dimensions is much more advanced than any…
Imagine instead the bitter lesson says: we can expand an outwards circle in many dimensions of ways to continuously mathematically manipulate data to adjust outputs.
Even the attention-token approach is on the grand scale of things a simple line outwards from the centre; we have not even explored around the centre (with the same compute spend) for things like non-token generation, different layers/different activation functions and norming / query/key/value set up (why do we only use the 3 inherent to contextualising tokens, why not add a 4th matrix for something else?), character, sentence, whole thought, paragraph one-shot generation, positional embeddings which could work differently.
The bitter lesson says there is almost a work completely untouched by our findings for us to explore. The temporary work of non-data approaches can piggy back off a point on the line; it cannot expand it like we can as we exude out from the circle..
Re: Learning to Reason with LLMs
#420A near perfect on AMC 12, 1900 CodeForces ELO, and silver medal IOI competitor. In two years, we'll have models that could easily win IMO and IOI. This is __incredible__!!
Near perfect AIME, not just AMC12.
But each solve costs far more time and energy than a competent human takes.