Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

621–630 of 1001 posts

Re: Learning to Reason with LLMs

#621

Post by ramesh31 »

lukev wrote: post #512

This is a pretty big technical achievement, and I am excited to see this type of advancement in the field. However, I am very worried about the utility of this tool given that it (like all LLMs) is still prone to hallucination. Exactly who is it for? If you're enough of an expert to critically judge the output, you're probably just as well off doing the reasoning yourself. If you're not capable of evaluating the outp…

>If you're enough of an expert to critically judge the output, you're probably just as well off doing the reasoning yourself.

Thought requires energy. A lot of it. Humans are for more efficient in this regard than LLMs, but then a bicycle is also much more efficient than a race car. I've found that even when they are hilariously wrong about something, simply the directionality of the line of reasoning can be enough to usefully accelerate my own thought.

Re: Learning to Reason with LLMs

#622

Post by kmeisthax »

>We believe that a hidden chain of thought presents a unique opportunity for monitoring models. Assuming it is faithful and legible, the hidden chain of thought allows us to "read the mind" of the model and understand its thought process. For example, in the future we may wish to monitor the chain of thought for signs of manipulating the user. However, for this to work the model must have freedom to express its thoughts in unaltered form, so we cannot train any policy compliance or user preferences onto the chain of thought. We also do not want to make an unaligned chain of thought directly visible to users.

>Therefore, after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users. We acknowledge this decision has disadvantages. We strive to partially make up for it by teaching the model to reproduce any useful ideas from the chain of thought in the answer. For the o1 model series we show a model-generated summary of the chain of thought.

So, let's recap. We went from:

- Weights-available research prototype with full scientific documentation (GPT-2)

- Commercial-scale model with API access only, full scientific documentation (GPT-3)

- Even bigger API-only model, tuned for chain-of-thought reasoning, minimal documentation on the implementation (GPT-4, 4v, 4o)

- An API-only model tuned to generate unedited chain-of-thought, which will not be shown to the user, even though it'd be really useful to have (o1)

Re: Learning to Reason with LLMs

#623

Post by mmckelvy »

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

As others have said, LLMs still require engineers to produce quality output. LLMs do, however, make those engineers that use them much more productive. If this trend continues, I could see a scenario where an individual engineer could build a customized version of, say, Salesforce in a month or two. If that happens, you could make a solid case that companies paying $1mm+ per year for 12 different SaaS tools should just bring that in house. The upshot is you may still be writing software, but instead of building SaaS at Salesforce, you'll be working for their former customers or maybe as some sort of contractor.

Re: Learning to Reason with LLMs

#624

Post by rovr138 »

avodonosov wrote: post #614
bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

The screenshot [1] is not readable for me. Chrome, Android. It's so blurry that I cant recognize a single character. How do other people read it? The resolution is 84x800.

When I click on the image, it expands to full res, 1713x16392.3

Re: Learning to Reason with LLMs

#625

Post by noshitsherlock »

This is great. I've been wondering how we will revert back to an agrarian society! You know, beating our swords into plowshares; more leisure time, visiting with good people, getting to know their thoughts hopes and dreams, playing music together, taking time contemplating the vastness and beauty of the universe. We're about to come full circle; back to Eden. It all makes sense now.

Re: Learning to Reason with LLMs

#627

Post by guiambros »

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

FYI, there's a "Save ChatGPT as PDF" Chrome extension [1].

I wouldn't use on a ChatGPT for Business subscription (it may be against your company's policies to export anything), but very convenient for personal use.

https://chromewebstore.google.com/detail/save-chatgpt-as-pdf...

Re: Learning to Reason with LLMs

#628

Post by mikebridgman »

avodonosov wrote: post #614
bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

The screenshot [1] is not readable for me. Chrome, Android. It's so blurry that I cant recognize a single character. How do other people read it? The resolution is 84x800.

Click on it for full resolution

Re: Learning to Reason with LLMs

#629

Post by hibikir »

ponector wrote: post #499
al_borland wrote: post #473

Earlier quoted context omitted.

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

But excel eliminated need in multiple accountants. One accountant with excel replaced ten with paper. Chatgpt already eliminated many entry-level jobs like writer or illustrator. Instead of hiring multiple teams of developers, there will be one team with few seniors and multiple AI coding tools. Guess how depressing to the IT salaries it will be?

A whole lot of automation is limited not by what could be automated, but what one can automate within a given budget.

When I was coding in the 90s, I was in a team that replaced function calls into new and exciting interactions with other computers which, using a queuing system, would do the computation and return the answer back. We'd have a project of having someone serialize the C data structures that were used on both sides into something that would be compatible, and could be inspected in the middle.

Today we call all of that a web service, the serialization would take a minute to code, and be doable by anyone. My entire team would be out of work! And yet, today we have more people writing code than ever.

When one accountant can do the work of 10 accountants, the price of the task lowers, but a lot of people that before couldn't afford accounting now can. And the same 10 accountaings from before can just do more work, and get paid about the same.

As far as software, we are getting paid A LOT more than in the early 90s. We are just doing things that back then would be impossible to pay for, our just outright impossible to do due to lack of compute capacity.

Re: Learning to Reason with LLMs

#630

Post by machiaweliczny »

gradus_ad wrote: post #117

Interesting sequence from the Cipher CoT: Third pair: 'dn' to 'i' 'd'=4, 'n'=14 Sum:4+14=18 Average:18/2=9 9 corresponds to 'i'(9='i') But 'i' is 9, so that seems off by 1. So perhaps we need to think carefully about letters. Wait, 18/2=9, 9 corresponds to 'I' So this works. ----- This looks like recovery from a hallucination. Is it realistic to expect CoT to be able to recover from hallucinations this quickly?

In general if hallucination ratio is 2% can't it be reduced to 0.04% by running twice or sth like this. I think they should try establishing the facts from different angles and this probably would work fine to minimize hallucinations. But if this was that simple somebody would already do it...
Post reply on HN