Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

631–640 of 1001 posts

Re: Learning to Reason with LLMs

#631

Post by m3kw9 »

islewis wrote: post #124

My first interpretation of this is that it's jazzed-up Chain-Of-Thought. The results look pretty promising, but i'm most interested in this: > Therefore, after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users. Mentioning competitive advantage here signals to me that OpenA…

The moat is expanding from use count, also the moat is to lead and advance faster than anyone can catch up, you will always have the best mode with the best infrastructure and low limits.

Re: Learning to Reason with LLMs

#635

Post by [deleted] »

ComputerGuru wrote: post #347

The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…

[deleted]

Re: Learning to Reason with LLMs

#636

Post by bunderbunder »

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

Actually cutting code is maybe 10% of the job, and LLMs are absolute crap at the other 90%.

They can't build and maintain relationships with stakeholders. They can't tell you why what you ask them to do is unlikely to work out well in practice and suggest alternative designs. They can't identify, document and justify acceptance criteria. They can't domain model. They can't architect. They can't do large-scale refactoring. They can't do system-level optimization. They can't work with that weird-ass code generation tool that some hotshot baked deeply into the system 15 years ago. They can't figure out why that fence is sitting out in the middle of the field for no obvious reason. etc.

If that kind of stuff sounds like satisfying work to you, you should be fine. If it sounds terrible, you should pivot away now regardless of any concerns about LLMs, because, again, this is like 90% of the real work.

Re: Learning to Reason with LLMs

#637

Post by taco_emoji »

spaceman_2020 wrote: post #479
canjobear wrote: post #427

First shot, I gave it a medium-difficulty math problem, something I actually wanted the answer to (derive the KL divergence between two Laplace distributions). It thought for a long time, and still got it wrong, producing a plausible but wrong answer. After some prodding, it revised itself and then got it wrong again. I still feel that I can't rely on these systems.

Look where you were 3 years ago, and where you are now. And then imagine where you will be in 5 more years. If it can almost get a complex problem right now, I'm dead sure it will get it correct within 5 years

Have you never heard of "local maxima"? Why are you so certain another 5 years will provide any qualitative advancement at all?

Re: Learning to Reason with LLMs

#638

Post by nickfromseattle »

patapong wrote: post #86

Very interesting. I guess this is the strawberry model that was rumoured. I am a bit surprised that this does not beat GPT-4o for personal writing tasks. My expectations would be that a model that is better at one thing is better across the board. But I suppose writing is not a task that generally requires "reasoning steps", and may also be difficult to evaluate objectively.

Maybe math is easier to score and do reinforcement learning on because of it's 'solvability' whereas writing requires human judgement to score?

Re: Learning to Reason with LLMs

#639

Post by dvt »

kmeisthax wrote: post #622

>We believe that a hidden chain of thought presents a unique opportunity for monitoring models. Assuming it is faithful and legible, the hidden chain of thought allows us to "read the mind" of the model and understand its thought process. For example, in the future we may wish to monitor the chain of thought for signs of manipulating the user. However, for this to work the model must have freedom to express its thoug…

It's clear to me that OpenAI is quickly realizing they have no moat. Even this obfuscation of the chain-of-thought isn't really a moat. On top of CoT being pretty easy to implement and tweak, there's a serious push to on-device inference (which imo is the future), so the question is: will GPT-5 and beyond be really that much better than what we can run locally?

Re: Learning to Reason with LLMs

#640

Post by xivzgrev »

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

Play it out

Let's assume today a LLM is perfectly equivalent to a junior software engineer. You connect it to your code base, load in PRDs / designs, ask it to build it, and viola perfect code files

1) Companies are going to integrate this new technology in stages / waves. It will take time for this to really get broad adoption. Maybe you are at the forefront of working with these models

2) OK the company adopts it and fires their junior engineers. They start deploying code. And it breaks Saturday evening. Who is going to fix it? Customers are pissed. So there's lots to work out around support.

3) That problem is solved, we can perfectly trust a LLM to ship perfect code that never causes downstream issues and perfectly predicts all user edge cases.

Never underestimate the power of corporate greediness. There's generally two phases of corporate growth - expansion and extraction. Expansion is when they throw costs out the window to grow. Extraction is when growth stops, and they squeeze customers & themselves.

AI is going to cause at least a decade of expansion. It opens up so many use cases that were simply not possible before, and lots of replacement.

Companies are probably not looking at their engineers looking to cut costs. They're more likely looking at them and saying "FINALLY, we can do MORE!"

You won't be a coder - you'll be a LLM manager / wrangler. You will be the neck the company can choke if code breaks.

Remember if a company can earn 10x money off your salary, it's a good deal to keep paying you.

Maybe some day down the line, they'll look to squeeze engineers and lay some off, but that is so far off.

This is not hopium, this is human nature. There's gold in them hills.

But you sure as shit better be well versed in AI and using in your workflows - the engineers who deny it will be the ones who fall behind

Post reply on HN