Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

911–920 of 1001 posts

Re: Learning to Reason with LLMs

#911

Post by crazygringo »

ActorNightly wrote: post #794
crazygringo wrote: post #310

Earlier quoted context omitted.

Seriously. I actually feel as impressed by the chain of thought, as I was when ChatGPT first came out. This isn't "just" autocompletion anymore, this is actual step-by-step reasoning full of ideas and dead ends and refinement, just like humans do when solving problems. Even if it is still ultimately being powered by "autocompletion". But then it makes me wonder about human reasoning, and what if it's similar? Just fo…

Again its not reasoning. Reasoning would imply that it can figure out stuff without being trained in it. The chain of thought is basically just a more accurate way to map input to output. But its still a map, i.e forward only. If an LLM coud reason, you should be able to ask it a question about how to make a bicycle frame from scratch with a small home cnc with limited work area and it should be able to iterate on an…

I think you might be confusing two concepts here.

It's definitely reasoning. We can watch that in action, whatever the mechanism behind it is.

But it's not doing long-term learning, it's not updating its model.

Re: Learning to Reason with LLMs

#912

Post by greenavocado »

philipkglass wrote: post #432
ComputerGuru wrote: post #347

The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…

If somebody needs step by step instructions from an LLM to synthesize strychnine, they don't have the practical laboratory skills to synthesize strychnine [1]. There's no increased real world risk of strychnine poisonings whether or not an LLM refuses to answer questions like that. However , journalists and regulators may not understand why superficially dangerous-looking instructions carry such negligible real world…

The kind of harm they are worried about stems from questioning the foundations of protected status for certain peoples from first principles and other problems which form identities of entire peoples. I can't be more specific without being banned here.

Re: Learning to Reason with LLMs

#913

Post by Emiledel »

baq wrote: post #662
activatedgeek wrote: post #610

Earlier quoted context omitted.

Reasoning tokens are indeed billed as output tokens. > While reasoning tokens are not visible via the API, they still occupy space in the model's context window and are billed as output tokens. From here: https://platform.openai.com/docs/guides/reasoning

This is concerning - how do you know you aren’t being fleeced out of your money here…? You’ll get your results, but did you really use that much?

In the UI the reasoning is visible. The API can probably return it too, just check the code

Re: Learning to Reason with LLMs

#914

Post by Zenzero »

danielmarkbruce wrote: post #886
mountainriver wrote: post #542

Earlier quoted context omitted.

It’s basically a scaled Tree of Thoughts

This seems most likely, with some special tokens thrown in to kick off different streams of thought.

To me it looks like they paired two instances of the model to feed off of each other's outputs with some sort of "contribute to reasoning out this problem" prompt. In the prior demos of 4o they did several similar demonstrations of that with audio.

Re: Learning to Reason with LLMs

#915

Post by haolez »

gwern wrote: post #887
haolez wrote: post #633

This should also be good news for open weights models, right? Since OpenAI is basically saying "you can get very far with good prompts and some feedback loops".

No. It's bad news, because you can't see the rationale/search process that led to the final answer, just the final answer, and if training on the final answer were really that adequate, we wouldn't be here. It also is probably massively expensive compute-wise, much more so than simple unsupervised training on a corpus of question/answer pairs (because you have to generate the corpus by search first). It's also also b…

> It also is probably massively expensive compute-wise, much more so than simple unsupervised training on a corpus of question/answer pairs (because you have to generate the corpus by search first).

What do you mean? It sounds interesting.

Re: Learning to Reason with LLMs

#916

Post by gen220 »

lukev wrote: post #907
ramesh31 wrote: post #621

Earlier quoted context omitted.

>If you're enough of an expert to critically judge the output, you're probably just as well off doing the reasoning yourself. Thought requires energy. A lot of it. Humans are for more efficient in this regard than LLMs, but then a bicycle is also much more efficient than a race car. I've found that even when they are hilariously wrong about something, simply the directionality of the line of reasoning can be enough t…

Look, I've been experimenting with this for the past year, and this is definitely the happy path. The unhappy path, which I've also experienced, is that the model outputs something plausible but false but that aligns with an area where my thinking was already confused and sends me down the wrong path. I've had to calibrate my level of suspicion, and so far using these things more effectively has always been in the di…

There are phases in every developer’s growth, where you transition from asking coworkers or classmates, to asking on stack overflow, to reading stack overflow, to reading docs and man pages and mailing lists and source code.

I think like you, I worry that LLMs will handicap this trajectory for people newer in the field, because GPT-4/Sonnet/Whatever are an exceptionally good classmate/coworker. So good that you might try to delay progressing along that trajectory.

But LLMs have all the flaws of a classmate: they aren’t authoritative, their opinions are strongly stated but often based on flimsy assumptions that you aren’t qualified to refute or verify, and so on.

I know intellectually that the kids will be alright, but it’ll be interesting to see how we get there. I suspect that as time goes on people will simply increase their discount rate on LLM responses, like you have, until they get dissatisfied with that value and just decide to get good at reading docs.

Re: Learning to Reason with LLMs

#917

Post by drewbeck »

jazzyjackson wrote: post #454
w4 wrote: post #439

Earlier quoted context omitted.

There are two basic versions of “safety” which are related, but distinct: One version of “safety” is a pernicious censorship impulse shared by many modern intellectuals, some of whom are in tech. They believe that they alone are capable of safely engaging with the world of ideas to determine what is true, and thus feel strongly that information and speech ought to be censored to prevent the rabble from engaging in wr…

Third version is "brand safety" which is, we don't want to be in a new york times feature about 13 year olds following anarchist-cookbook instructions from our flagship product

This is imo the most important one to the businesses creating these models and is way under appreciated. Folks who want a “censorship-free” model from businesses don’t understand what a business is for.

Re: Learning to Reason with LLMs

#918

Post by deeznuttynutz »

lopatamd wrote: post #508
al_borland wrote: post #473

Earlier quoted context omitted.

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

You can't ignore the fact that literally studying coding at this point is so demoralizing and you don't need really to study much if you think about it. You only need to be able to read the code to understand if it generated correctly etc but when if you don't understand some framework you just ask it to explain it to you etc. Basically gives vibes of a skill not being used anymore that much by us programmers. But wi…

I completed the book Programming Principles and Practice using C++ (which I HIGHLY recommend to any beginner interested in software engineering) about year ago with GPT4 as a companion. I read the book throughly and did all the exercises, only asking questions to GPT4 when I was stuck. This took me about 900-1000 hours total. Although I achieved my goal of learning C++ to a basic novice level, I acquired another skill unintentionally: the ability to break down tasks effectively to LLMs and prompt in a fashion that is extremely modular. I've been able to create complex apps and programs in a variety of programming languages even though I really only know C++. It has been an eye-opening experience. Of course it isn't perfect, but it is mind blowing and quite disturbing.

Re: Learning to Reason with LLMs

#919

Post by gdhkgdhkvff »

ActorNightly wrote: post #794
crazygringo wrote: post #310

Earlier quoted context omitted.

Seriously. I actually feel as impressed by the chain of thought, as I was when ChatGPT first came out. This isn't "just" autocompletion anymore, this is actual step-by-step reasoning full of ideas and dead ends and refinement, just like humans do when solving problems. Even if it is still ultimately being powered by "autocompletion". But then it makes me wonder about human reasoning, and what if it's similar? Just fo…

Again its not reasoning. Reasoning would imply that it can figure out stuff without being trained in it. The chain of thought is basically just a more accurate way to map input to output. But its still a map, i.e forward only. If an LLM coud reason, you should be able to ask it a question about how to make a bicycle frame from scratch with a small home cnc with limited work area and it should be able to iterate on an…

It’s like every single sentence you just wrote is incorrect?

1. You’re making up some weird goalposts here of what it means to reason. It’s not reasoning unless it can access the internet to search for parts? No. That has nothing to do with reasoning. You just think it would be cool if it could do that.

2. “Can figure out stuff without being trained on it” That’s exactly what it’s doing in the cypher example. It wasn’t trained to know that that input meant the corresponding output through the cypher. Emergent reasoning through autocomplete, sure, but that’s still reasoning.

3. “Forward only”. If that was the case then back and forth conversations with the llm would be pointless. It wouldn’t be able to improve upon previous answers it gave you when you give it new details. But that’s not how it works. If tell it one thing, then separately tell it another thing, it can change its original conclusion based on your new input.

4. Even desolate your convoluted test for reasoning, ChatGPT CAN do what you asked… even using the internet to look up parts it can either do out of the box or could do if given a plug-in to allow that.

Re: Learning to Reason with LLMs

#920

Post by pizza »

philipkglass wrote: post #432
ComputerGuru wrote: post #347

The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…

If somebody needs step by step instructions from an LLM to synthesize strychnine, they don't have the practical laboratory skills to synthesize strychnine [1]. There's no increased real world risk of strychnine poisonings whether or not an LLM refuses to answer questions like that. However , journalists and regulators may not understand why superficially dangerous-looking instructions carry such negligible real world…

I remember Dario Amodei mentioned in a podcast once that most models won't tell you the practical lab skills you need. But that sufficiently-capable models would and do tell you the practical lab skills (without your needing to know to ask it to in the first place), in addition to the formal steps.
Post reply on HN