Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

571–580 of 1001 posts

Re: Learning to Reason with LLMs

#571

Post by fsndz »

lumost wrote: post #526
cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

LLMs perform well on small tasks that are well defined. This definition matches almost every task that a student will work on in school leading to an overestimation of LLM capabiity. LLMs cannot decide what to work on, or manage large bodies of work/code easily. They do not understand the risk of making a change and deploying it to production, or play nicely in autonomous settings. There is going to be a massive amou…

Exactly, LLMs are not near ready to fully replace software engineers or any kind of knowledge workers. But they are increasingly useful tools that is true. https://www.lycee.ai/blog/ai-replace-software-engineer

Re: Learning to Reason with LLMs

#572

Post by klabb3 »

jazzyjackson wrote: post #454
w4 wrote: post #439

Earlier quoted context omitted.

There are two basic versions of “safety” which are related, but distinct: One version of “safety” is a pernicious censorship impulse shared by many modern intellectuals, some of whom are in tech. They believe that they alone are capable of safely engaging with the world of ideas to determine what is true, and thus feel strongly that information and speech ought to be censored to prevent the rabble from engaging in wr…

Third version is "brand safety" which is, we don't want to be in a new york times feature about 13 year olds following anarchist-cookbook instructions from our flagship product

And the fourth version, which is investor-regulator safety mid point: so capable and dangerous that competitors shouldn’t even be allowed to research it, but just safe enough that only our company is responsible enough to continue mass commercial consumer deployment without any regulations at all. It’s a fine line.

Re: Learning to Reason with LLMs

#574

Post by fshbbdssbbgdd »

ComputerGuru wrote: post #347

The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…

So if there’s already easily available information about strychnine, that makes it a good example to use for the demo, because you can safely share the demo and you aren’t making the problem worse.

On the other hand, suppose there are other dangerous things, where the information exists in some form online, but not packaged together in an easy to find and use way, and your model is happy to provide that. You may want to block your model from doing that (and brag about it, to make sure everyone knows you’re a good citizen who doesn’t need to be regulated by the government), but you probably wouldn’t actually include that example in your demo.

Re: Learning to Reason with LLMs

#575

Post by zmgsabst »

vasco wrote: post #510
al_borland wrote: post #473

Earlier quoted context omitted.

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

Accounting mechanization is a good example of how unpredictable it can be. Initially there were armies of "accountants" (what we now call bookkeepers), mostly doing basic tasks of collecting data and making it fit something useful. When mechanization appeared, the profession split into bookkeeping and accounting. Bookkeeping became a job for women as it was more boring and could be paid lower salaries (we're in the 1…

We’re already seeing that split, between “developer” and “engineer”. We have been for years.

But that’s normal, eg, we have different standards for a shed (yourself), house (carpenter and architect), and skyscraper (bonded firms and certified engineers).

Re: Learning to Reason with LLMs

#577

Post by [deleted] »

ponector wrote: post #499
al_borland wrote: post #473

Earlier quoted context omitted.

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

But excel eliminated need in multiple accountants. One accountant with excel replaced ten with paper. Chatgpt already eliminated many entry-level jobs like writer or illustrator. Instead of hiring multiple teams of developers, there will be one team with few seniors and multiple AI coding tools. Guess how depressing to the IT salaries it will be?

[deleted]

Re: Learning to Reason with LLMs

#578

Post by morningsam »

What sticks out to me is the 60% win rate vs GPT-4o when it comes to actual usage by humans for programming tasks. So in reality it's barely better than GPT-4o. That the figure is higher for mathematical calculation isn't surprising because LLMs were much worse at that than at programming to begin with.

Re: Learning to Reason with LLMs

#579

Post by noshitsherlock »

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

There's so much software yet to to be written, so much to automate, so many niches to attack that you need not worry. It takes humans to know where to apply the technology based on their heart, not brains. Use AI in the direction only you can ascertain; and do it for the good of HUMANITY. It's a tool that makes the knowledge posterity has left us accessible, like mathematics. Go forth an conquer life's ills young man; It takes a human to know one. Don't worry, you're created in God's image.

Re: Learning to Reason with LLMs

#580

Post by PoignardAzur »

Jasper_ wrote: post #171
Hansenq wrote: post #87

Reading through the Chain of Thought for the provided Cipher example (go to the example, click "Show Chain of Thought") is kind of crazy...it literally spells out every thinking step that someone would go through mentally in their head to figure out the cipher (even useless ones like "Hmm"!). It really seems like slowing down and writing down the logic it's using and reasoning over that makes it better at logic, simi…

> Average:18/2=9 > 9 corresponds to 'i'(9='i') > But 'i' is 9, so that seems off by 1. Still seems bad at counting, as ever.

It's interesting that it makes that mistake, but then catches it a few lines later.

A common complaint about LLMs is that once they make a mistake, they will keep making it and write the rest of their completion under the assumption that everything before was correct. Even if they've been RLHF to take human feedback into account and the human points out the mistake, their answer is "Certainly! Here's the corrected version" and then they write something that makes the same mistake.

So it's interesting that this model does something that appears to be self-correction.

Post reply on HN