Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

561–570 of 1001 posts

Re: Learning to Reason with LLMs

#561

Post by mikeyouse »

ponector wrote: post #499
al_borland wrote: post #473

Earlier quoted context omitted.

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

But excel eliminated need in multiple accountants. One accountant with excel replaced ten with paper. Chatgpt already eliminated many entry-level jobs like writer or illustrator. Instead of hiring multiple teams of developers, there will be one team with few seniors and multiple AI coding tools. Guess how depressing to the IT salaries it will be?

I don’t don’t doubt that it might depress salaries but that excel example is a good one in that suddenly every company could start to do basic financial analysis in a manner that only the largest ones could previously afford.

Re: Learning to Reason with LLMs

#562

Post by rcarmo »

fwip wrote: post #557
rcarmo wrote: post #533

Here's an unpopular take on this: "We had the chance to make AI decision-making auditable but are locking ourselves out of hundreds of critical applications by not exposing the chain of thought." One of the key blockers in many customer discussions I have is that AI models are not really auditable and that automating complex processes with them (let alone debug things when "reasoning" goes awry) is difficult if not i…

I suspect that actually reading the "chain of thought" would reveal obvious "logic" errors embarrassingly often.

It would still be auditable. In a few industries that is the only blocker for adoption--even if the outputs are incorrect.

Re: Learning to Reason with LLMs

#563

Post by geenkeuse »

Average Joe's like myself will build our apps end to end with the help of AI.

The only shops left standing will be Code Auditors.

The solopreneur will wing it, without them, but enterprises will take the (very expensive) hit to stay safe and compliant.

Everyone else needs to start making contingency plans.

Magnus Carlsen is the best chess player in the world, but he is not arrogant enough to think he can go head to head with Stockfish and not get a beating.

Re: Learning to Reason with LLMs

#564

Post by derefr »

philipkglass wrote: post #432
ComputerGuru wrote: post #347

The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…

If somebody needs step by step instructions from an LLM to synthesize strychnine, they don't have the practical laboratory skills to synthesize strychnine [1]. There's no increased real world risk of strychnine poisonings whether or not an LLM refuses to answer questions like that. However , journalists and regulators may not understand why superficially dangerous-looking instructions carry such negligible real world…

I would think that the risk isn’t of a human being reading those instructions, but of those instructions being automatically piped into an API request to some service that makes chemicals on demand and then sends them by mail, all fully automated with no human supervision.

Not that there is such a service… for chemicals. But there do exist analogous systems, like a service that’ll turn whatever RNA sequence you send it into a viral plasmid and encapsulate it helpfully into some E-coli, and then mail that to you.

Or, if you’re working purely in the digital domain, you don’t even need a service. Just show the thing the code of some Linux kernel driver and ask it to discover a vuln in it and generate code to exploit it.

(I assume part of the thinking here is that these approaches are analogous, so if they aren’t unilaterally refusing all of them, you could potentially talk the AI around into being okay with X by pointing out that it’s already okay with Y, and that it should strive to hold to a consistent/coherent ethics.)

Re: Learning to Reason with LLMs

#565

Post by blibble »

epcoa wrote: post #546
cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

The “progress” demonstrated in this example is to literally just extract bytes from the middle of a number: Is this task: “About 2 minutes later, these values were captured, again spaced 5 seconds apart. 0160093201 0160092d01 0160092801 0160092301 0160091e01” [Find the part that is changing] really even need an AI to assist (this should be a near instant task for a human with basic CS numerical skills)? If this is th…

I put his value into my hex editor and it instantly showed 900 in the data inspector pane

Re: Learning to Reason with LLMs

#566

Post by fsndz »

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

> Asking the models to determine if my code is equivalent to what they reverse engineered resulted in a nuanced and thorough examination, and eventual conclusion that it is equivalent.

Did you actually implement to see if it works out of the box ?

Also if you are a free users or accepted that your chats should be used for training then maybe o1 is was just trained on your previous chat and so now knows how to reason about that particular type of problems

Re: Learning to Reason with LLMs

#567

Post by baq »

margorczynski wrote: post #548
baq wrote: post #498

Earlier quoted context omitted.

If you’re any good at SWE with a sprinkle of math and CS, your advantage will get multiplied by anywhere from 2 to 100x if you use the leverage of co-intelligence correctly. Things that took weeks before now easily take hours, so if you know what to build and especially what not to build (including but not limited to confabulations of models), you’ll do well.

But also on the other hand you'll need much less people to achieve the same effect. Effectively a whole team could be replaced by one lead guy that just based on the requirements orders the LLM what to do and glues it together.

Yes - my point is: be that guy

Re: Learning to Reason with LLMs

#568

Post by mmckelvy »

refulgentis wrote: post #459
cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

I went from economics dropout waiter who built a app startup with $0 funding and $1M a year in revenue by midway through year 1, sold it a few years later, then went to Google for 7 years, and last year I left. I'm mentioning that because the following sounds darn opinionated and brusque without the context I've capital-S seen a variety of people and situations. Sit down and be really honest with yourself. If your go…

What did your startup do?

Re: Learning to Reason with LLMs

#569

Post by squigz »

w4 wrote: post #439
ComputerGuru wrote: post #347

The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…

There are two basic versions of “safety” which are related, but distinct: One version of “safety” is a pernicious censorship impulse shared by many modern intellectuals, some of whom are in tech. They believe that they alone are capable of safely engaging with the world of ideas to determine what is true, and thus feel strongly that information and speech ought to be censored to prevent the rabble from engaging in wr…

Whether you agree with the lengths that are gone to or not, 'safety' in this space is a very real concern, and simply reciting information as in GP's example is only 1 part of it. In my experience, people who think it's all about "censorship" and handwave it away tend to be very ideologically driven.

Re: Learning to Reason with LLMs

#570

Post by deisteve »

spaceman_2020 wrote: post #505
deisteve wrote: post #27

Earlier quoted context omitted.

so o1 seems like it has real measurable edge, crushing it in every single metric, i mean 1673 elo is insane, and 89th percentile is like a whole different league, and it looks like it's not just a one off either, it's consistently performing way better than gpt-4o across all the datasets, even in the ones where gpt-4o was already doing pretty well, like math and mmlu, o1 is just taking it to the next level, and the f…

1673 ELO is wild If its actually true in practice, I sincerely cannot imagine a scenario where it would be cheaper to hire actual junior or mid-tier developers (keyword: "developers", not architects or engineers). 1,673 ELO should be able to build very complex, scalable apps with some guidance

currently my workflow is generate some code, run it, if it doesn't run i tell LLM what I expected, it will then produce code and I frequently tell it how to reason about the problem.

with O1 being in the 89th percentile would mean it should be able to think at junior to intermediate level with very strong consistency.

i dont think people in the comments realize the implication of this. previously LLMs were able to only "pattern match" but now its able to evaluate itself (with some guidance ofc) essentially, steering the software into depth of edge cases and reason about it in a way that feels natural to us.

currently I'm copying and pasting stuff and notifying LLM the results but once O1 is available its going to significantly lower that frequency.

For example, I expect it to self evaluate the code its generate and think at higher levels.

ex) oooh looks like this user shouldn't be able to escalate privileges in this case because it would lead to security issues or it could conflict with the code i generated 3 steps ago, i'll fix it myself.

Post reply on HN