Learning to Reason with LLMs • Hacker News Board

Re: Learning to Reason with LLMs

#281

Post by thomasahle » Thu, Sep 12, 2024, 6:21 PM UTC

csomar wrote: post #266

I gave the Crossword puzzle to Claude and got a correct response[1]. The fact that they are comparing this to gpt4o and not to gpt4 suggests that it is less impressive than they are trying to pretend. [1]: Based on the given clues, here's the solved crossword puzzle: +---+---+---+---+---+---+ | E | S | C | A | P | E | +---+---+---+---+---+---+ | S | E | A | L | E | R | +---+---+---+---+---+---+ | T | E | R | E | S |…

As good as Claude has gotten recently in reasoning, they are likely using RL behind the scenes too. Supposedly, o1/strawberry was initially created as an engine for high-quality synthetic reasoning data for the new model generation. I wonder if Anthropic could release their generator as a usable model too.

Re: Learning to Reason with LLMs

#282

Post by [deleted] » Thu, Sep 12, 2024, 6:21 PM UTC

[deleted]

Re: Learning to Reason with LLMs

#283

Post by seydor » Thu, Sep 12, 2024, 6:21 PM UTC

asadm wrote: post #107

I am not up-to-speed on CoT side but is this similar to how perplexity does it ie. - generate a plan - execute the steps in plan (search internet, program this part, see if it is compilable) each step is a separate gpt inference with added context from previous steps. is O1 same? or does it do all this in a single inference run?

that is the summary of the task it presents to the user. The full chain of thought seems more mechanistic

Re: Learning to Reason with LLMs

#284

Post by mintone » Thu, Sep 12, 2024, 6:22 PM UTC

This video[1] seems to give some insight into what the process actually is, which I believe is also indicated by the output token cost.

Whereas GPT-4o spits out the first answer that comes to mind, o1 appears to follow a process closer to coming up with an answer, checking whether it meets the requirements and then revising it. The process of saying to an LLM "are you sure that's right? it looks wrong" and it coming back with "oh yes, of course, here's the right answer" is pretty familiar to most regular users, so seeing it baked into a model is great (and obviously more reflective of self-correcting human thought)

[1] https://vimeo.com/1008704043

Re: Learning to Reason with LLMs

#285

Post by tslater2006 » Thu, Sep 12, 2024, 6:23 PM UTC

Looking at pricing, its $15 per 1M input tokens, and $60 per 1M output tokens. I assume the CoT tokens count as output (or input even)? If so and it directly affects billing, I'm not sure how I feel about them hiding the CoT prompts. Nothing to stop them from saying "trust me bro, that used 10,000 tokens ok?". Also no way to gauge expected costs if there's a black box you are being charged for.

Re: Learning to Reason with LLMs

#286

Post by tedsanders » Thu, Sep 12, 2024, 6:24 PM UTC

theLiminator wrote: post #166

notamy wrote: post #46

https://openai.com/index/introducing-openai-o1-preview/ > ChatGPT Plus and Team users will be able to access o1 models in ChatGPT starting today. Both o1-preview and o1-mini can be selected manually in the model picker, and at launch, weekly rate limits will be 30 messages for o1-preview and 50 for o1-mini. We are working to increase those rates and enable ChatGPT to automatically choose the right model for a given p…

Anyone know when o1 access in ChatGPT will be open?

Rolling out over the next few hours to Plus users.

Re: Learning to Reason with LLMs

#287

Post by bigyikes » Thu, Sep 12, 2024, 6:24 PM UTC

gradus_ad wrote: post #117

Interesting sequence from the Cipher CoT: Third pair: 'dn' to 'i' 'd'=4, 'n'=14 Sum:4+14=18 Average:18/2=9 9 corresponds to 'i'(9='i') But 'i' is 9, so that seems off by 1. So perhaps we need to think carefully about letters. Wait, 18/2=9, 9 corresponds to 'I' So this works. ----- This looks like recovery from a hallucination. Is it realistic to expect CoT to be able to recover from hallucinations this quickly?

4o could already recover from hallucination in a limited capacity.

I’ve seen it, mid-reply say things like “Actually, that’s wrong, let me try again.”

Re: Learning to Reason with LLMs

#288

Post by cja » Thu, Sep 12, 2024, 6:24 PM UTC

cal85 wrote: post #54

Sounds great, but so does their "new flagship model that can reason across audio, vision, and text in real time" announced in May. [0] [0] https://openai.com/index/hello-gpt-4o/

Recently I was starting to think I imagined that. Back then they gave me the impression it would be released within week or so of the announcement. Have they explained the delay?

Re: Learning to Reason with LLMs

#289

Post by chairhairair » Thu, Sep 12, 2024, 6:25 PM UTC

RandomLensman wrote: post #218

How could it fail to solve some maths problems if it has a method for reasoning through things?

Simple questions like this are not welcomed by LLM hype sellers.

The word "reasoning" is being used heavily in this announcement, but with an intentional corruption of the normal meaning.

The models are amazing but they are fundamentally not "reasoning" in a way we'd expect a normal human to.

This is not a "distinction without a difference". You still CANNOT rely on the outputs of these models in the same way you can rely on the outputs of simple reasoning.

Re: Learning to Reason with LLMs

#290

Post by cs702 » Thu, Sep 12, 2024, 6:25 PM UTC

Before commenting here, please take 15 minutes to read through the chain-of-thought examples -- decoding a cypher-text, coding to solve a problem, solving a math problem, solving a crossword puzzle, answering a complex question in English, answering a complex question in Chemistry, etc.

After reading through the examples, I am shocked at how incredibly good the model is (or appears to be) at reasoning: far better than most human beings.

I'm impressed. Congratulations to OpenAI!