Learning to Reason with LLMs • Hacker News Board

Re: Learning to Reason with LLMs

#421

Post by bartman » Thu, Sep 12, 2024, 7:18 PM UTC

losvedir wrote: post #383

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

Wow, that is impressive! How were you able to use o1-preview? I pay for ChatGPT, but on chatgpt.com in the model selector I only see 4o, 4o-mini, and 4. Is o1 in that list for you, or is it somewhere else?

Like others here, it was just available on the website and app when I checked. FWIW I still don’t have advanced voice mode.

Re: Learning to Reason with LLMs

#422

Post by zh3 » Thu, Sep 12, 2024, 7:19 PM UTC

Question here is about the "reasoning" tag - behind the scenes, is this qualitively different fron stringing words together on a statistical basis? (aside from backroom tweaking and some randomisation).

Re: Learning to Reason with LLMs

#423

Post by hidelooktropic » Thu, Sep 12, 2024, 7:19 PM UTC

carabiner wrote: post #359

tylervigen wrote: post #293

Here's the o1-preview answer to the strawberry question: -- There are *three* letter "R"s in the word "strawberry." Let's break down the word to count the occurrences: - *S* - *T* - *R* - *A* - *W* - *B* - *E* - *R* - *R* - *Y* The letter "R" appears in positions 3, 8, and 9.

can you ask it: 9.11 and 9.9, which number is larger

> Evaluating Number formats > I'm considering that the question might be about comparing two numbers, 9.11 and 9.9, in different contexts such as different locale formats or representing dates. 9.11 is larger than 9.9

Well, it's kind of right because we left it ambiguous what kinds of numbers these are. September 11 is a greater date than September 9. However, I would want the model to have the street smarts to assume without further context, I must be just talking about numbers as in numbers.

Re: Learning to Reason with LLMs

#424

Post by rahimnathwani » Thu, Sep 12, 2024, 7:19 PM UTC

losvedir wrote: post #383

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

Wow, that is impressive! How were you able to use o1-preview? I pay for ChatGPT, but on chatgpt.com in the model selector I only see 4o, 4o-mini, and 4. Is o1 in that list for you, or is it somewhere else?

I think they're rolling it out gradually today. I don't see it listed (in the browser, Mac app or Android app).

Re: Learning to Reason with LLMs

#425

Post by wnolens » Thu, Sep 12, 2024, 7:20 PM UTC

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

If you're the type of person who isn't scared away easily by rapidly changing technology.

Re: Learning to Reason with LLMs

#426

Post by lupire » Thu, Sep 12, 2024, 7:20 PM UTC

seydor wrote: post #300

billconan wrote: post #72

I will pay if O1 can become my college level math tutor.

Looking at the full chain of thought , it involves a lot of backtracking and even hallucination. It will be like a math teacher that is perpetually drunk and on speed

That's Paul Erdös

Re: Learning to Reason with LLMs

#427

Post by canjobear » Thu, Sep 12, 2024, 7:20 PM UTC

First shot, I gave it a medium-difficulty math problem, something I actually wanted the answer to (derive the KL divergence between two Laplace distributions). It thought for a long time, and still got it wrong, producing a plausible but wrong answer. After some prodding, it revised itself and then got it wrong again. I still feel that I can't rely on these systems.

Re: Learning to Reason with LLMs

#428

Post by romeros » Thu, Sep 12, 2024, 7:21 PM UTC

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

is it better than Claude?

Re: Learning to Reason with LLMs

#429

Post by antman » Thu, Sep 12, 2024, 7:21 PM UTC

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

second is very blurry

Re: Learning to Reason with LLMs

#430

Post by hidelooktropic » Thu, Sep 12, 2024, 7:22 PM UTC

packetlost wrote: post #177

lol at the graphs at the top. Logarithmic scaling for test/compute time should make everyone who thinks AGI is possible with this architecture take pause.

I don't see any log scaled graphs.