Learning to Reason with LLMs • Hacker News Board

Re: Learning to Reason with LLMs

#501

Post by vanviegen » Thu, Sep 12, 2024, 7:49 PM UTC

armchairhacker wrote: post #462

crazygringo wrote: post #310

Earlier quoted context omitted.

Seriously. I actually feel as impressed by the chain of thought, as I was when ChatGPT first came out. This isn't "just" autocompletion anymore, this is actual step-by-step reasoning full of ideas and dead ends and refinement, just like humans do when solving problems. Even if it is still ultimately being powered by "autocompletion". But then it makes me wonder about human reasoning, and what if it's similar? Just fo…

I think it's similar, although I think it would be more similar if the LLM did the steps in lower layers (not in English), and instead of the end being fed to the start, there would be a big mess of cycles throughout the neural net. That could be more efficient since the cycles are much smaller, but harder to train.

It doesn't do the 'thinking' in English (inference is just math), but it does now verbalize intermediate thoughts in English (or whatever the input language is, presumably), just like humans tend to do.

Re: Learning to Reason with LLMs

#502

Post by retrofuturism » Thu, Sep 12, 2024, 7:50 PM UTC

bevenky wrote: post #315

For folks who want to see some demo videos and be amazed! HTML Snake - https://vimeo.com/1008703890 Video Game Coding - https://vimeo.com/1008704014 Coding - https://youtu.be/50W4YeQdnSg?si=IohJlJNY-WS394uo Counting - https://vimeo.com/1008703993 Korean Cipher - https://vimeo.com/1008703957 Devin AI founder - https://vimeo.com/1008674191 Quantum Physics - https://vimeo.com/1008662742 Math - https://vimeo.com/10087041…

In "HTML Snake" the video cuts just as the snake intersects with the obstacle. Presumably because the game crashed (I can't see endGame defined anywhere)

This video is featured in the main announcement so it's kinda dishonest if you ask me.

Re: Learning to Reason with LLMs

#503

Post by MattGaiser » Thu, Sep 12, 2024, 7:50 PM UTC

ponector wrote: post #499

al_borland wrote: post #473

Earlier quoted context omitted.

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

But excel eliminated need in multiple accountants. One accountant with excel replaced ten with paper. Chatgpt already eliminated many entry-level jobs like writer or illustrator. Instead of hiring multiple teams of developers, there will be one team with few seniors and multiple AI coding tools. Guess how depressing to the IT salaries it will be?

Accountants still make plenty of money. Expertise in Excel also pays well independently of that.

Re: Learning to Reason with LLMs

#504

Post by packetlost » Thu, Sep 12, 2024, 7:50 PM UTC

Workaccount2 wrote: post #483

hakanderyal wrote: post #418

Earlier quoted context omitted.

Software engineering contains a lot more than just writing code. If we somehow get AGI, it'll change everything, not just SWE. If not, my belief is that there will be a lot more demand for good SWEs to harness the power of LLMs, not less. Use them to get better at it faster.

I don't think anyone is worried about SWE work going away, I think the concern is if SWE's will still be able to command cushy salaries and working conditions.

I think the industry will bifurcate along the axis of "doing actually novel stuff" vs slinging DB records and displaying web pages. The latter is what I'd expect to get disrupted, if anything, but the former isn't going away unless real AGI is created. The people on the left of that split are going to be worth a lot more because the pipeline to get there will be even harder than it was before.

Re: Learning to Reason with LLMs

#505

Post by spaceman_2020 » Thu, Sep 12, 2024, 7:50 PM UTC

deisteve wrote: post #27

farresito wrote: post #3

Damn, that looks like a big jump.

so o1 seems like it has real measurable edge, crushing it in every single metric, i mean 1673 elo is insane, and 89th percentile is like a whole different league, and it looks like it's not just a one off either, it's consistently performing way better than gpt-4o across all the datasets, even in the ones where gpt-4o was already doing pretty well, like math and mmlu, o1 is just taking it to the next level, and the f…

1673 ELO is wild

If its actually true in practice, I sincerely cannot imagine a scenario where it would be cheaper to hire actual junior or mid-tier developers (keyword: "developers", not architects or engineers).

1,673 ELO should be able to build very complex, scalable apps with some guidance

Re: Learning to Reason with LLMs

#506

Post by sterlind » Thu, Sep 12, 2024, 7:50 PM UTC

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

Because none of your other majors will hold up much longer. Once software engineering becomes fully automated, so will EE, ME, applied math, economics, physics, etc. If you work with your hands, like a surgeon or chemist, you'll last longer, but the thinky bits of those jobs will disappear. And once AI research is automated, how long will it be until we have dexterous robots?

So basically, switching majors is just running to the back of a sinking ship. Sorry.

Re: Learning to Reason with LLMs

#507

Post by reliabilityguy » Thu, Sep 12, 2024, 7:50 PM UTC

w4 wrote: post #439

ComputerGuru wrote: post #347

The "safety" example in the "chain-of-thought" widget/preview in the middle of the article is absolutely ridiculous. Take a step back and look at what OpenAI is saying here "an LLM giving detailed instructions on the synthesis of strychnine is unacceptable, here is what was previously generated vs our preferred, neutered content " What's this obsession with "safety" when it comes to LLMs? "This knowledge is perfectly…

There are two basic versions of “safety” which are related, but distinct: One version of “safety” is a pernicious censorship impulse shared by many modern intellectuals, some of whom are in tech. They believe that they alone are capable of safely engaging with the world of ideas to determine what is true, and thus feel strongly that information and speech ought to be censored to prevent the rabble from engaging in wr…

> In order for AIs to fit into our society and behave ethically they need to know how to flag that thought as a bad idea and not act on it.

Don’t you think that by just parsing the internet and the classical literature, the LLM would infer on its own that poisoning someone to solve a problem is not okay?

I feel that in the end the only way the “safety” is introduced today is by censoring the output.

Re: Learning to Reason with LLMs

#508

Post by lopatamd » Thu, Sep 12, 2024, 7:51 PM UTC

al_borland wrote: post #473

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

You can't ignore the fact that literally studying coding at this point is so demoralizing and you don't need really to study much if you think about it. You only need to be able to read the code to understand if it generated correctly etc but when if you don't understand some framework you just ask it to explain it to you etc. Basically gives vibes of a skill not being used anymore that much by us programmers. But will shift in more prompting and verifying and testing

Re: Learning to Reason with LLMs

#509

Post by baq » Thu, Sep 12, 2024, 7:51 PM UTC

cs391231 wrote: post #440

crop_rotation wrote: post #434

Earlier quoted context omitted.

If you have better career ideas, you should not continue. The thing is it is very hard to predict how the world will change (and by how much from very little to a revolutionary change) with all these new changes. Only licensed and regulated professions (doctors/lawyers/pilots etc) might remain high earning for long (and they too are not guaranteed). It really is worth a relook on what you want to do in life while see…

I don't have any ideas whatsoever.

Do you enjoy making computers solve problems? If yes, continue. If you hate it and are in just for the money… I’d say flip a coin.

Re: Learning to Reason with LLMs

#510

Post by vasco » Thu, Sep 12, 2024, 7:51 PM UTC

al_borland wrote: post #473

cs391231 wrote: post #408

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

Accounting mechanization is a good example of how unpredictable it can be. Initially there were armies of "accountants" (what we now call bookkeepers), mostly doing basic tasks of collecting data and making it fit something useful.

When mechanization appeared, the profession split into bookkeeping and accounting. Bookkeeping became a job for women as it was more boring and could be paid lower salaries (we're in the 1800s here). Accountants became more sophisticated but lower numbers as a %. Together, both professions grew like crazy in total number though.

So if the same happens you could predict a split between software engineers and prompt engineers. With an explosion in prompt engineers paid much less than software engineers.

> the number of accountants/book- keepers in the U.S. increased from circa 54,000 workers [U.S. Census Office, 1872, p. 706] to more than 900,000 [U.S. Bureau of the Census, 1933, Tables 3, 49].

> These studies [e.g., Coyle, 1929; Baker, 1964; Rotella, 1981; Davies, 1982; Lowe, 1987; DeVault, 1990; Fine, 1990; Strom, 1992; Kwolek-Folland, 1994; Wootton and Kemmerer, 1996] have traced the transformation of the of- fice workforce (typists, secretaries, stenographers, bookkeepers) from predominately a male occupation to one primarily staffed by women, who were paid substantially lower wages than the men they replaced.

> Emergence of mechanical accounting in the U.S., 1880-1930 [PDF download] https://www.google.com/url?sa=t&source=web&rct=j&opi=8997844...