Earlier quoted context omitted.
Why not? Isn't that basically what humans do? Sit there and think for a while before answering, going down different branches/chains of thought?
Sure, but if I want a human, I can hire a human. Humans also do many other things I don't want my LLM to do.
Learning to Reason with LLMs
641–650 of 1001 posts
Re: Learning to Reason with LLMs
#642The request is pretty basic. If anyone can get it to work, I'd like to know how and what model you're using. I tried it with gpt4o1 and after ~10 iterations of showing it the failed output, it still failed to come up with a one-line command to properly display results.
Here it what I asked: Using a mac osx terminal and standard available tools, provide a command to update the output of netstat -an to show the fqdn of IP addresses listed in the result.
This is what it came up with:
netstat -an | awk '{for(i=1;i4?"."a[5]:"");cmd="dig +short -x "ip;cmd|getline h;close(cmd);if(h){sub(/\.$/,"",h);$i=h port}}}}1'
Re: Learning to Reason with LLMs
#643Re: Learning to Reason with LLMs
#644Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
There is little to no research that shows modern AI can perform even the most simple long-running task without training data on that exact problem. To my knowledge, there is no current AI system that can replace a white collar worker in any multistep task. The only thing they can do is support the worker. Most jobs are safe for the forseable future. If your job is highly repetitive and a company can produce a perfect…
Re: Learning to Reason with LLMs
#645>We believe that a hidden chain of thought presents a unique opportunity for monitoring models. Assuming it is faithful and legible, the hidden chain of thought allows us to "read the mind" of the model and understand its thought process. For example, in the future we may wish to monitor the chain of thought for signs of manipulating the user. However, for this to work the model must have freedom to express its thoug…
It's clear to me that OpenAI is quickly realizing they have no moat. Even this obfuscation of the chain-of-thought isn't really a moat. On top of CoT being pretty easy to implement and tweak, there's a serious push to on-device inference (which imo is the future), so the question is: will GPT-5 and beyond be really that much better than what we can run locally?
Re: Learning to Reason with LLMs
#646My point of view: this is a real advancement. I’ve always believed that with the right data allowing the LLM to be trained to imitate reasoning, it’s possible to improve its performance. However, this is still pattern matching, and I suspect that this approach may not be very effective for creating true generalization. As a result, once o1 becomes generally available, we will likely notice the persistent hallucinatio…
I had been using 4o as a rubber ducky for some projects recently. Since I appeared to have access to o1-preview, I decided to go back and redo some of those conversations with o1-preview.
I think your comment is spot on. It's definitely an advancement, but still makes some pretty clear mistakes and does some fairly faulty reasoning. It especially seems to have a hard time with causal ordering, and reasoning about dependencies in a distributed system. Frequently it gets the relationships backwards, leading to hilarious code examples.
Re: Learning to Reason with LLMs
#647Re: Learning to Reason with LLMs
#648Re: Learning to Reason with LLMs
#649Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?
I agree there's too much cope going around. All the people saying AI is just a tool to augment our jobs are correct, humans are still needed but perhaps far less of them will be needed. If job openings shrink by 50% or disproportionately impact juniors it will hurt. One decent reason to continue is that pretty much all white collar professions will be impacted by this. I think it's a big enough number that the powers…