Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

861–870 of 1001 posts

Re: Learning to Reason with LLMs

#861

Post by FridgeSeal »

ARandumGuy wrote: post #137

One thing that makes me skeptical is the lack of specific labels on the first two accuracy graphs. They just say it's a "log scale", without giving even a ballpark on the amount of time it took. Did the 80% accuracy test results take 10 seconds of compute? 10 minutes? 10 hours? 10 days? It's impossible to say with the data they've given us. The coding section indicates "ten hours to solve six challenging algorithmic…

Bold of you to expect transparency and clarity from a company like OpenAI.

You wanted reliable readable graphs? Ppphhh, get out of here, but pay of for the CoT tokens you’ll never see on your way out though.

Re: Learning to Reason with LLMs

#862

Post by matwood »

UncleOxidant wrote: post #551
cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

Semi-retired software/hardware engineer here. After my recent experiences with various coding LLMs (similar to the experience of the OP with the bluetooth fan protocol) I'm really glad I'm in a financial position such that I'm able to retire. The progress of these LLMs at coding has been astonishing over the last 18 months. Will they entirely replace humans? No. But as they increase programmer productivity fewer devs…

> But as they increase programmer productivity fewer devs will be required.

Can you point me to any company whose feature pipeline is finite? Maybe these tools will help us reach that point, but every company I've ever worked for, and every person I know who works in tech has a backlog that is effectively infinite at this point.

Maybe if only a few companies had access to coding LLMs they could cut their stuff, when the whole industry raises the bar, nothing really changes.

Re: Learning to Reason with LLMs

#863

Post by silveryfu »

After playing with it on ChatGPT this morning, it seems a reasonable strategy of using the o1 model is to:

- If your request requires reasoning, switch to o1 model.

- If not, switch to 4o model.

This applies to both across chat sessions and within the same session (yes, we can switch between models within the same session and it looks like down the road OpenAI is gonna support automatic model switching). Based on my experience, this will actually improve the perceived response quality -- o1 and 4o are rather complementary to each other rather than replacement.

Re: Learning to Reason with LLMs

#865

Post by piker »

joshhug wrote: post #854

I just tried o1, and it did pretty well with understanding this minor issue with subtitles on a Dutch TV show we were watching. I asked it "I was watching a show and in the subtitles an umlaut u was rendered as 1/4, i.e. a single character that said 1/4. Why would this happen?" and it gave a pretty thorough explanation of exactly which encoding issue was to blame. https://chatgpt.com/share/66e37145-72bc-800a-be7b-f7c…

A common problem, no doubt, with a lot of training context. But man. What a time to be alive.

Re: Learning to Reason with LLMs

#866

Post by thimabi »

fzaninotto wrote: post #438

It can solve sudoku. It took 119s to solve this easy grid: _ 7 8 4 1 _ _ _ 9 5 _ 1 _ 2 _ 4 7 _ _ 2 9 _ 6 _ _ _ _ _ 3 _ _ _ 7 6 9 4 _ 4 5 3 _ _ 8 1 _ _ _ _ _ _ _ 3 _ _ 9 _ 4 6 7 2 1 3 _ 6 _ _ _ _ _ 7 _ 8 _ _ _ 8 3 1 _ _ _

I tried to have it solve an easy Sudoku grid too, but in my case it failed miserably. It kept making mistakes and saying that there was a problem with the puzzle (there wasn’t).

Re: Learning to Reason with LLMs

#867

Post by matwood »

spaceman_2020 wrote: post #602
lumost wrote: post #526

Earlier quoted context omitted.

LLMs perform well on small tasks that are well defined. This definition matches almost every task that a student will work on in school leading to an overestimation of LLM capabiity. LLMs cannot decide what to work on, or manage large bodies of work/code easily. They do not understand the risk of making a change and deploying it to production, or play nicely in autonomous settings. There is going to be a massive amou…

Careers are 30 years long Can you confidently say that an LLM won’t be better than an average 22 year old coder within these 30 years?

22 year old coder today or 22 year old coder 30 years from now? How a 22 year old codes 30 years from now may look like magic to you and me.

Re: Learning to Reason with LLMs

#868

Post by breadwinner »

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

From NYT article on this model: "The chatbot also answered a Ph.D.-level chemistry question and diagnosed an illness based on a detailed report about a patient’s symptoms and history."

So it is not just software engineering, it is also chemistry and even medicine. Every science and art major should consider whether they should quit school. Ultimately the answer is no, don't quit school because AI makes us productive, and that will make everything cheaper, but will not eliminate the need for humans. Hopefully.

Re: Learning to Reason with LLMs

#869

Post by OutOfHere »

delusional wrote: post #786

Great, yet another step towards the inevitable conclusion. Now I'm not just being asked to outsource my thinking to my computer, but instead to a black box operated by a for-profit company for the benefit of Microsoft. Not only will they not tell me the whole reasoning chain, they wont even tell me how they came up with it. Tell me, users of this tool. What's even are you? If you've outsourced your thinking to a corp…

> What's even are you?

An accelerator of reaching the Singularity. This is something more.

Re: Learning to Reason with LLMs

#870

Post by zamadatix »

silveryfu wrote: post #863

After playing with it on ChatGPT this morning, it seems a reasonable strategy of using the o1 model is to: - If your request requires reasoning, switch to o1 model. - If not, switch to 4o model. This applies to both across chat sessions and within the same session (yes, we can switch between models within the same session and it looks like down the road OpenAI is gonna support automatic model switching). Based on my…

Given the rate limits are 30 reqs/week most probably want to start with:

- Try it a bit with 4o, see if you're getting anywhere

- Switch to the new o1 model if it's just not working out, take your improved base prompt and follow ups with you so it only counts as 1 req

Post reply on HN