Learning to Reason with LLMs • Hacker News Board

Re: Learning to Reason with LLMs

#541

Post by joshstrange » Thu, Sep 12, 2024, 7:59 PM UTC

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

Software engineering teaches you a set of skills that are applicable in more places than just writing software. There are big parts of the job that cannot be done by LLMs (today) and if LLMs get better (or AGI happens) then enough other professions will be affected that we will all be in the same boat (no matter what you major in).

LLMs are just tools, they help but they do not replace developers (yet).

Re: Learning to Reason with LLMs

#542

Post by mountainriver » Thu, Sep 12, 2024, 7:59 PM UTC

utdiscant wrote: post #326

Feels like a lot of commenters here miss the difference between just doing chain-of-thought prompting, and what is happening here, which is learning a good chain of thought strategy using reinforcement learning. "Through reinforcement learning, o1 learns to hone its chain of thought and refine the strategies it uses." When looking at the chain of thought (COT) in the examples, you can see that the model employs diffe…

It’s basically a scaled Tree of Thoughts

Re: Learning to Reason with LLMs

#543

Post by colonelspace » Thu, Sep 12, 2024, 7:59 PM UTC

spaceman_2020 wrote: post #479

canjobear wrote: post #427

First shot, I gave it a medium-difficulty math problem, something I actually wanted the answer to (derive the KL divergence between two Laplace distributions). It thought for a long time, and still got it wrong, producing a plausible but wrong answer. After some prodding, it revised itself and then got it wrong again. I still feel that I can't rely on these systems.

Look where you were 3 years ago, and where you are now. And then imagine where you will be in 5 more years. If it can almost get a complex problem right now, I'm dead sure it will get it correct within 5 years

> I'm dead sure it will get it correct within 5 years

You might be right.

But plenty of people said we'd all be getting around in self-driving cars for sure 10 years ago.

Re: Learning to Reason with LLMs

#544

Post by fshbbdssbbgdd » Thu, Sep 12, 2024, 8:00 PM UTC

jazzyjackson wrote: post #448

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

Isn't there a big "Share" button at the top right of the chatgpt interface? Or are you using another front end?

I often click on those links and get an error that they are unavailable. I’m not sure if it’s openAI trying to prevent people from sharing evidence of the model behaving badly, or an innocuous explanation like the links are temporary.

Re: Learning to Reason with LLMs

#545

Post by soheil » Thu, Sep 12, 2024, 8:00 PM UTC

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

Great progress, I asked GPT-4o and o1-preview to create a python script to make $100 quickly, o1 came up with a very interesting result:

https://x.com/soheil/status/1834320893331587353

Re: Learning to Reason with LLMs

#546

Post by epcoa » Thu, Sep 12, 2024, 8:00 PM UTC

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

The “progress” demonstrated in this example is to literally just extract bytes from the middle of a number:

Is this task:

“About 2 minutes later, these values were captured, again spaced 5 seconds apart.

0160093201 0160092d01 0160092801 0160092301 0160091e01”

[Find the part that is changing]

really even need an AI to assist (this should be a near instant task for a human with basic CS numerical skills)? If this is the type of task one thinks an AI would be useful for they are likely in trouble for other reasons.

Also notable that you can cherry pick more impressive feats even from older models, so I don’t necessarily think this proves progress.

I still wouldn’t get too carried away just yet.

Re: Learning to Reason with LLMs

#547

Post by dymk » Thu, Sep 12, 2024, 8:01 PM UTC

Jasper_ wrote: post #171

Hansenq wrote: post #87

Reading through the Chain of Thought for the provided Cipher example (go to the example, click "Show Chain of Thought") is kind of crazy...it literally spells out every thinking step that someone would go through mentally in their head to figure out the cipher (even useless ones like "Hmm"!). It really seems like slowing down and writing down the logic it's using and reasoning over that makes it better at logic, simi…

> Average:18/2=9 > 9 corresponds to 'i'(9='i') > But 'i' is 9, so that seems off by 1. Still seems bad at counting, as ever.

The next line is it catching its own mistake, and noting i = 9.

Re: Learning to Reason with LLMs

#548

Post by margorczynski » Thu, Sep 12, 2024, 8:01 PM UTC

baq wrote: post #498

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

If you’re any good at SWE with a sprinkle of math and CS, your advantage will get multiplied by anywhere from 2 to 100x if you use the leverage of co-intelligence correctly. Things that took weeks before now easily take hours, so if you know what to build and especially what not to build (including but not limited to confabulations of models), you’ll do well.

But also on the other hand you'll need much less people to achieve the same effect. Effectively a whole team could be replaced by one lead guy that just based on the requirements orders the LLM what to do and glues it together.

Re: Learning to Reason with LLMs

#549

Post by Vecr » Thu, Sep 12, 2024, 8:01 PM UTC

Ninjinka wrote: post #32

Someone give this model an IQ test stat.

It can't do large portions of the parts of an IQ test (not multi-modal). Otherwise I think it's essentially superhuman, modulo tokenization issues (please start running byte-by-byte or at least come up with a better tokenizer).

Re: Learning to Reason with LLMs

#550

Post by legel » Thu, Sep 12, 2024, 8:01 PM UTC

seydor wrote: post #259

afro88 wrote: post #186

Earlier quoted context omitted.

Seeing the "hmmm", "perfect!" etc. one can easily imagine the kind of training data that humans created for this. Being told to literally speak their mind as they work out complex problems.

looks a bit like 'code', using keywords 'Hmm', 'Alternatively', 'Perfect'

As a technical engineer, I’ve learned the value of starting sentences with “basically”, even when I’m facing technical uncertainty. Basically, “basically” forces me to be simple.

Being trained to say words like “Alternatively”, “But…”, “Wait!”, “So,” … based on some metric of value in focusing / switching elsewhere / … is basically brilliant.