Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

611–620 of 1001 posts

Re: Learning to Reason with LLMs

#611

Post by amrrs »

OkGoDoIt wrote: post #522

Some practical notes from digging around in their documentation: In order to get access to this, you need to be on their tier 5 level, which requires $1,000 total paid and 30+ days since first successful payment. Pricing is $15.00 / 1M input tokens and $60.00 / 1M output tokens. Context window is 128k token, max output is 32,768 tokens. There is also a mini version with double the maximum output tokens (65,536 tokens…

The CoT is billed as output tokens. Mentioned in the docs where it talks about reasoning

Re: Learning to Reason with LLMs

#612

Post by HappMacDonald »

spaceman_2020 wrote: post #602
lumost wrote: post #526

Earlier quoted context omitted.

LLMs perform well on small tasks that are well defined. This definition matches almost every task that a student will work on in school leading to an overestimation of LLM capabiity. LLMs cannot decide what to work on, or manage large bodies of work/code easily. They do not understand the risk of making a change and deploying it to production, or play nicely in autonomous settings. There is going to be a massive amou…

Careers are 30 years long Can you confidently say that an LLM won’t be better than an average 22 year old coder within these 30 years?

Careers have failed to be 30 years long for a lot longer than 30 years now. That's one of the reasons that 4-year colleges have drastically lost their ROI, the other blade of those scissors being the stupendously rising tuition. AI is nothing but one more layer in the constantly growing substrate of computing technology a coder has to learn how to integrate into their toolbelts. Just like the layers that came before it: mobile, virtualization, networking, etc.

Re: Learning to Reason with LLMs

#613

Post by morningsam »

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

As soon as software development can be fully performed by AIs, it won't take long before all other jobs that can be performed in front of a computer follow, and after that it probably won't take long for practically the entire rest.

This release has shifted my personal prediction of when this is going to happen further into the future, because OpenAI made a big deal hyping it up and it's nothing - preferred by humans over GPT-4o only a little more than half the time.

Re: Learning to Reason with LLMs

#614

Post by avodonosov »

bartman wrote: post #367

This is incredible. In April I used the standard GPT-4 model via ChatGPT to help me reverse engineer the binary bluetooth protocol used by my kitchen fan to integrate it into Home Assistant. It was helpful in a rubber duck way, but could not determine the pattern used to transmit the remaining runtime of the fan in a certain mode. Initial prompt here [0] I pasted the same prompt into o1-preview and o1-mini and both c…

The screenshot [1] is not readable for me. Chrome, Android. It's so blurry that I cant recognize a single character. How do other people read it? The resolution is 84x800.

Re: Learning to Reason with LLMs

#615

Post by _proofs »

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

just because something can generate an output for you, does not make a need for discernment and application obsolete.

like another commenter, i do not have a lot of faith, that people who do not have at minimum: fundamental fluency in programming (even with a dash of general software architecture and practices).

there is no "push button generate and glueing components together in a way that can survive at scale and be maintainable" without knowing what the output means, and implies with respect to integration(s).

however, those with the fluency, domain, and experience, will thrive, and continue thriving.

Re: Learning to Reason with LLMs

#617

Post by m3kw9 »

canjobear wrote: post #427

First shot, I gave it a medium-difficulty math problem, something I actually wanted the answer to (derive the KL divergence between two Laplace distributions). It thought for a long time, and still got it wrong, producing a plausible but wrong answer. After some prodding, it revised itself and then got it wrong again. I still feel that I can't rely on these systems.

Maybe you are wrong if you don’t know the answer?

Re: Learning to Reason with LLMs

#618

Post by patapong »

slashdave wrote: post #384
patapong wrote: post #86

Very interesting. I guess this is the strawberry model that was rumoured. I am a bit surprised that this does not beat GPT-4o for personal writing tasks. My expectations would be that a model that is better at one thing is better across the board. But I suppose writing is not a task that generally requires "reasoning steps", and may also be difficult to evaluate objectively.

> My expectations would be that a model that is better at one thing is better across the board. No, it's the opposite. This is simply a function of resources applied during training.

To some extent I agree, but until now all of the big jumps (GPT2 -> GPT3 -> GPT4) have meant significant improvements across all tasks. This does not seem to be the case here, this model seems to be vastly stronger on certain tasks but not much of an improvement on other tasks. Maybe we will have to wait for GPT5 for that :)

Re: Learning to Reason with LLMs

#619

Post by patapong »

afro88 wrote: post #232
patapong wrote: post #86

Very interesting. I guess this is the strawberry model that was rumoured. I am a bit surprised that this does not beat GPT-4o for personal writing tasks. My expectations would be that a model that is better at one thing is better across the board. But I suppose writing is not a task that generally requires "reasoning steps", and may also be difficult to evaluate objectively.

The solution of the cipher example problem also strongly hints at this: "there are three r's in strawberry"

Confirmed by the verge: https://www.theverge.com/2024/9/12/24242439/openai-o1-model-...

Re: Learning to Reason with LLMs

#620

Post by taco_emoji »

spaceman_2020 wrote: post #602
lumost wrote: post #526

Earlier quoted context omitted.

LLMs perform well on small tasks that are well defined. This definition matches almost every task that a student will work on in school leading to an overestimation of LLM capabiity. LLMs cannot decide what to work on, or manage large bodies of work/code easily. They do not understand the risk of making a change and deploying it to production, or play nicely in autonomous settings. There is going to be a massive amou…

Careers are 30 years long Can you confidently say that an LLM won’t be better than an average 22 year old coder within these 30 years?

yes, because this is still glorified autocomplete
Post reply on HN